FSM Week 5 Update

Progress

After the meeting, Robin and me had a conversation about the benchmark. We were both confused about what benchmarks are and what specific benchmarks in our project right now. If I understood correctly, Robin was thinking about what to evaluate if we use the similar benchmark in ML: are we evaluating the performance of the final actions by the robot or the FSM generated by LLM. For benchmark for the robot, we think that’s something alreadly been discussed. But for benchmarking FSM, we need to do some investigations.

It just brings me to some broader questions:

• What are benchmarks?
• Why do we need a benchmark?
• What do we care about when setting up benchmarks?

And I found this article that talks about benchmarking. It mentions a key point that we need to create a simple benchmark in the very beginning of our project, and the goal is to track our progress.

Habitat Installation Walkthrough

Successfully set up Habitat locally. Spent over 10 hours working on the whole setup, there were lots of small bugs during the process. Follow the steps below and you should successfully complete the setup. Helped Robin set up the local Habitatl environment successfully.

Running Enviorment: Macbook Pro M1 Chip, MacOS Sonoma 14.1 Habitat Version: 3.0, Habitat Sim, Habitat Lab

# Install Conda, not miniconda
CONDA_SUBDIR=osx-64 conda create -n habitat python=3.9 cmake=3.14.0 
# use higher cmake version if possible
conda activate habitat
 
conda install habitat-sim withbullet -c conda-forge -c aihabitat
git clone --branch stable  https://github.com/facebookresearch/habitat-lab.git
cd habitat-lab
 
pip install -e habitat-lab
pip install -e habitat-baselines
conda update --all
 
git lfs install
# if you don’t have lfs
python -m habitat_sim.utils.datasets_download --uids habitat_test_scenes --data-path data/
python -m habitat_sim.utils.datasets_download --uids habitat_test_pointnav_dataset --data-path data/
#if you see some packages missing try conda uninstall that package, then use conda install again

Errors Handling: MP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized.

# run this before your first Habitat running
export KMP_DUPLICATE_LIB_OK=True

Related Links

• Useful installation walkthrough: https://zhuanlan.zhihu.com/p/672898454 , in Chinese
• Habitat Sim repo: https://github.com/facebookresearch/habitat-sim
• Habitat Lab repo: https://github.com/facebookresearch/habitat-lab