Sim to real gap
Hi Saurabh, we have an openarm in our office and want to deploy a model to test it. we were thinking if we can get away without fine tuning something from scratch. If my understanding is correct, you are using a dataset you created in isaac sim. Did you deploy your model to a physical robot? Was there a sim to real gap?
Hi there
The finetuned model that I trained using dataset created on isaac sim haven't been deployed and tested on the real physical robot so I am not sure about sim to real gap there.
btw in my opinion, your thoughts on running the model directly on a real robots won't perform as expected , I trying this same thing in simulation and it didn't worked out at all.
Think of finetuning process as a robot actuators calibrations for the model.
Hey Saurabh, wanted to get more insight on how you collected data in issac sim? Currently trying to built a quest 3 webXR bridge to the open arm and getting some issues. wondering if you were open to connect more privately and discuss more about this?
Hi Ammar ,
Initially I also faced issue while trying to find the tool through which I can record the dataset in Isaac sim for openarm , So created my own data collection pipeline using meta quest 3 for openarm. Currently I am captureing the controllers movement using webxr and relaying it to my local pc via websocket and dynamically converting them into ROS topics which becomes input for isaac sim running on a remote server. I have opensourced the entire data collection pipeline so check that out , see if it helps you.
link : https://github.com/AiSaurabhPatil/quest3_streamer
I am also open to connect privately if you want more discussion on it , here is my email : saurabhpatil1956@gmail.com