Skip to content

Commit ba77fbe

Browse files
goldbattlegoldbattle
authored
Merge pull request #117 from rpng/develop_v2.3
Development v2.3 - Memory Management, Active Feature Publishing, Small Fixes
2 parents e3cc117 + 0114e8e commit ba77fbe

File tree

98 files changed

+2986
-1935
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

98 files changed

+2986
-1935
lines changed

‎.github/workflows/build_catkin.yml

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: C/C++ CI
2+
3+
on:
4+
push:
5+
branches: [ master, develop_v2.3 ]
6+
pull_request:
7+
branches: [ master, develop_v2.3 ]
8+
9+
jobs:
10+
build:
11+
runs-on: ${{ matrix.os }}
12+
if: "!contains(github.event.head_commit.message, 'skip ci')"
13+
strategy:
14+
fail-fast: false
15+
matrix:
16+
os: [ubuntu-16.04, ubuntu-18.04, ubuntu-20.04]
17+
compiler: ["/usr/bin/g++"]
18+
steps:
19+
- uses: actions/checkout@v2
20+
- name: Checkout submodules
21+
uses: textbook/git-checkout-submodule-action@master
22+
23+
- name: Install apt dependencies (ubuntu-16.04)
24+
if: matrix.os == 'ubuntu-16.04'
25+
run: |
26+
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list' &&
27+
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 &&
28+
curl -sSL 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654' | sudo apt-key add - &&
29+
sudo apt update && sudo apt -y install build-essential ros-kinetic-desktop python-catkin-pkg python-catkin-tools libeigen3-dev git &&
30+
touch ~/.bashrc && echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc
31+
32+
- name: Install apt dependencies (ubuntu-18.04)
33+
if: matrix.os == 'ubuntu-18.04'
34+
run: |
35+
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list' &&
36+
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 &&
37+
curl -sSL 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654' | sudo apt-key add - &&
38+
sudo apt update && sudo apt -y install build-essential ros-melodic-desktop python-catkin-pkg python-catkin-tools libeigen3-dev git &&
39+
touch ~/.bashrc && echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
40+
41+
- name: Install apt dependencies (ubuntu-20.04)
42+
if: matrix.os == 'ubuntu-20.04'
43+
run: |
44+
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list' &&
45+
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 &&
46+
curl -sSL 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654' | sudo apt-key add - &&
47+
sudo apt update && sudo apt -y install build-essential ros-noetic-desktop python3-catkin-pkg python3-catkin-tools python3-osrf-pycommon libeigen3-dev git &&
48+
touch ~/.bashrc && echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
49+
50+
- name: Create catkin workspace and compile
51+
run: |
52+
export REPO=$(basename $GITHUB_REPOSITORY) &&
53+
cd $GITHUB_WORKSPACE/.. && mkdir src/ &&
54+
mv $REPO/ src/ && mkdir $REPO/ && mv src/ $REPO/ && cd $REPO/ &&
55+
source /opt/ros/$(ls -1 /opt/ros/ | head -n1)/setup.bash && echo "ros version: $ROS_DISTRO" &&
56+
catkin build -j2 --no-status
57+
58+
- name: Run simulation dataset
59+
run: |
60+
source $GITHUB_WORKSPACE/devel/setup.bash &&
61+
roscore &
62+
export REPO=$(basename $GITHUB_REPOSITORY) &&
63+
source $GITHUB_WORKSPACE/devel/setup.bash &&
64+
rosrun ov_msckf run_simulation _sim_traj_path:=$GITHUB_WORKSPACE/src/$REPO/ov_data/sim/udel_gore.txt

‎.gitignore

+3-1
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,6 @@ build
33
Build
44
*.*~
55
.idea
6-
doxgen_generated
6+
doxgen_generated
7+
*.swp
8+
*.swo

‎ReadMe.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
1-
2-
31
# OpenVINS
2+
![C/C++ CI](https://github.com/rpng/open_vins/workflows/C/C++%20CI/badge.svg)
43

54

65
Welcome to the OpenVINS project!
@@ -19,6 +18,7 @@ Please take a look at the feature list below for full details on what the system
1918

2019
## News / Events
2120

21+
* **December 1, 2020** - Released improved memory management, active feature pointcloud publishing, limiting number of features in update to bound compute, and other small fixes. See v2.3 [PR#117](https://github.com/rpng/open_vins/pull/117) for details.
2222
* **November 18, 2020** - Released groundtruth generation utility package, [vicon2gt](https://github.com/rpng/vicon2gt) to enable creation of groundtruth trajectories in a motion capture room for evaulating VIO methods.
2323
* **July 7, 2020** - Released zero velocity update for vehicle applications and direct initialization when standing still. See [PR#79](https://github.com/rpng/open_vins/pull/79) for details.
2424
* **May 18, 2020** - Released secondary pose graph example repository [ov_secondary](https://github.com/rpng/ov_secondary) based on [VINS-Fusion](https://github.com/HKUST-Aerial-Robotics/VINS-Fusion). OpenVINS now publishes marginalized feature track, feature 3d position, and first camera intrinsics and extrinsics. See [PR#66](https://github.com/rpng/open_vins/pull/66) for details and discussion.

‎docs/dev-profiling.dox

+157
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
/**
2+
3+
4+
@page dev-profiling System Profiling
5+
@tableofcontents
6+
7+
8+
@section dev-profiling-compute Profiling Processing Time
9+
10+
One way (besides inserting timing statements into the code) is to leverage a profiler such as [valgrind](https://www.valgrind.org/).
11+
This tool allows for recording of the call stack of the system.
12+
To use this with a ROS node, we can do the following (based on [this](http://wiki.ros.org/roslaunch/Tutorials/Roslaunch%20Nodes%20in%20Valgrind%20or%20GDB) guide):
13+
14+
- Edit `roslaunch ov_msckf pgeneva_serial_eth.launch` launch file
15+
- Append `launch-prefix="valgrind --tool=callgrind --callgrind-out-file=/tmp/callgrind.txt"` to your ROS node. This will cause the node to run with valgrind.
16+
- Change the bag length to be only 10 or so seconds (since profiling is slow)
17+
18+
@code{.shell-session}
19+
sudo apt install valgrind
20+
roslaunch ov_msckf pgeneva_serial_eth.launch
21+
@endcode
22+
23+
After running the profiling program we will want to visualize it.
24+
There are some good tools for that, specifically we are using [gprof2dot](https://github.com/jrfonseca/gprof2dot) and [xdot.py](https://github.com/jrfonseca/xdot.py).
25+
First we will post-process it into a xdot graph format and then visualize it for inspection.
26+
27+
@image html example_callgrind.png width=80%
28+
29+
@code{.shell-session}
30+
// install viz programs
31+
apt-get install python3 graphviz
32+
apt-get install gir1.2-gtk-3.0 python3-gi python3-gi-cairo graphviz
33+
pip install gprof2dot xdot
34+
// actually process and then viz call file
35+
gprof2dot --format callgrind --strip /tmp/callgrind.txt --output /tmp/callgrind.xdot
36+
xdot /tmp/callgrind.xdot
37+
@endcode
38+
39+
40+
41+
@section dev-profiling-leaks Memory Leaks
42+
43+
One can leverage a profiler such as [valgrind](https://www.valgrind.org/) to perform memory leak check of the codebase.
44+
Ensure you have installed the `valgrind` package (see above).
45+
We can change the node launch file as follows:
46+
47+
- Edit `roslaunch ov_msckf pgeneva_serial_eth.launch` launch file
48+
- Append `launch-prefix="valgrind --tool=memcheck --leak-check=yes"` to your ROS node. This will cause the node to run with valgrind.
49+
- Change the bag length to be only 10 or so seconds (since profiling is slow)
50+
51+
52+
This [page](https://web.stanford.edu/class/archive/cs/cs107/cs107.1206/resources/valgrind.html) has some nice support material for FAQ.
53+
An example loss is shown below which was found by memcheck.
54+
55+
@code{.text}
56+
==5512== 1,578,860 (24 direct, 1,578,836 indirect) bytes in 1 blocks are definitely lost in loss record 6,585 of 6,589
57+
==5512== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
58+
....
59+
==5512== by 0x543F868: operator[] (unordered_map.h:973)
60+
==5512== by 0x543F868: ov_core::TrackKLT::feed_stereo(double, cv::Mat&, cv::Mat&, unsigned long, unsigned long) (TrackKLT.cpp:165)
61+
==5512== by 0x4EF8C52: ov_msckf::VioManager::feed_measurement_stereo(double, cv::Mat&, cv::Mat&, unsigned long, unsigned long) (VioManager.cpp:245)
62+
==5512== by 0x1238A9: main (ros_serial_msckf.cpp:247)
63+
@endcode
64+
65+
66+
67+
68+
69+
70+
71+
@section dev-profiling-compiler Compiler Profiling
72+
73+
Here is a small guide on how to perform compiler profiling for building of the codebase.
74+
This should be used to try to minimize compile times which in general hurt developer productivity.
75+
It is recommended to read the following pages which this is a condenced form of:
76+
77+
- https://aras-p.info/blog/2019/01/16/time-trace-timeline-flame-chart-profiler-for-Clang/
78+
- https://aras-p.info/blog/2019/09/28/Clang-Build-Analyzer/
79+
80+
First we need to ensure we have a compiler that can profile the build time.
81+
Clang greater then 9 should work, but we have tested only with 11.
82+
We can get the [latest Clang](https://apt.llvm.org/) by using the follow auto-install script:
83+
84+
85+
@code{.shell-session}
86+
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
87+
export CC=/usr/bin/clang-11
88+
export CXX=/usr/bin/clang++-11
89+
@endcode
90+
91+
We then need to clone the analyzer repository, which allows for summary generation.
92+
93+
@code{.shell-session}
94+
git clone https://github.com/aras-p/ClangBuildAnalyzer
95+
cd ClangBuildAnalyzer
96+
cmake . && make
97+
@endcode
98+
99+
We can finally build our ROS package and time how long it takes.
100+
Note that we are using [catkin tools](https://catkin-tools.readthedocs.io/en/latest/) to build here.
101+
The prefix *CBA* means to run the command in the ClangBuildAnalyzer repository clone folder.
102+
While the prefix *WS* means run in the root of your ROS workspace.
103+
104+
@code{.shell-session}
105+
(WS) cd ~/workspace/
106+
(WS) catkin clean -y && mkdir build
107+
(CBA) ./ClangBuildAnalyzer --start ~/workspace/build/
108+
(WS) export CC=/usr/bin/clang-11 && export CXX=/usr/bin/clang++-11
109+
(WS) catkin build ov_msckf -DCMAKE_CXX_FLAGS="-ftime-trace"
110+
(CBA) ./ClangBuildAnalyzer --stop ~/workspace/build/ capture_file.bin
111+
(CBA) ./ClangBuildAnalyzer --analyze capture_file.bin > timing_results.txt
112+
@endcode
113+
114+
The `time-trace` flag should generate a bunch of .json files in your build folder.
115+
These can be opened in your chrome browser `chrome://tracing` for viewing.
116+
In general the ClangBuildAnalyzer is more useful for finding what files take long.
117+
An example output of what is generated in the timing_results.txt file is:
118+
119+
120+
@code{.text}
121+
Analyzing build trace from 'capture_file.bin'...
122+
**** Time summary:
123+
Compilation (86 times):
124+
Parsing (frontend): 313.9 s
125+
Codegen & opts (backend): 222.9 s
126+
127+
**** Files that took longest to parse (compiler frontend):
128+
13139 ms: /build//ov_msckf/CMakeFiles/ov_msckf_lib.dir/src/update/UpdaterSLAM.cpp.o
129+
12843 ms: /build//ov_msckf/CMakeFiles/run_serial_msckf.dir/src/ros_serial_msckf.cpp.o
130+
...
131+
132+
**** Functions that took longest to compile:
133+
1639 ms: main (/src/open_vins/ov_eval/src/error_comparison.cpp)
134+
1337 ms: ov_core::BsplineSE3::get_acceleration(double, Eigen::Matrix<double, ... (/src/open_vins/ov_core/src/sim/BsplineSE3.cpp)
135+
1156 ms: ov_eval::ResultSimulation::plot_state(bool, double) (/src/open_vins/ov_eval/src/calc/ResultSimulation.cpp)
136+
...
137+
138+
*** Expensive headers:
139+
27505 ms: /src/open_vins/ov_core/src/track/TrackBase.h (included 12 times, avg 2292 ms), included via:
140+
TrackKLT.cpp.o TrackKLT.h (4372 ms)
141+
TrackBase.cpp.o (4297 ms)
142+
TrackSIM.cpp.o TrackSIM.h (4252 ms)
143+
...
144+
@endcode
145+
146+
Some key methods to reduce compile times are as follows:
147+
- Only include headers that are required for your class
148+
- Don't include headers in your header files `.h` that are only required in your `.cpp` source files.
149+
- Consider [forward declarations](https://www.wikiwand.com/en/Forward_declaration) of methods and types
150+
- Ensure you are using an include guard in your headers
151+
152+
153+
154+
155+
156+
157+
*/

‎docs/dev-welcome.dox

+1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
- @subpage dev-docs --- Developer guide on how documentation can be built
88
- @subpage dev-index --- Description of the covariance index system
99
- @subpage dev-roadmap --- Where we plan to go in the future
10+
- @subpage dev-profiling --- Some notes on performing profiling
1011

1112

1213

‎docs/eval-error.dox

+1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33

44
@page eval-error Filter Error Evaluation Methods
5+
@tableofcontents
56

67

78
@m_class{m-note m-warning}

‎docs/eval-metric.dox

+1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33

44
@page eval-metrics Filter Evaluation Metrics
5+
@tableofcontents
56

67

78

‎docs/eval-timing.dox

+1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33

44
@page eval-timing Filter Timing Analysis
5+
@tableofcontents
56

67

78
@m_class{m-note m-warning}

‎docs/fej.dox

+1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33

44
@page fej First Estimate Jacobian (FEJ) Estimation
5+
@tableofcontents
56

67

78

‎docs/gs-installing.dox

+25-17
Original file line numberDiff line numberDiff line change
@@ -6,39 +6,48 @@
66

77
@section gs-install-ros ROS Dependency
88

9-
Our codebase is built on top of the [Robot Operating System (ROS)](https://www.ros.org/) and has been tested building on Ubuntu 16.04 systems with ROS Kinetic Kame.
9+
Our codebase is built on top of the [Robot Operating System (ROS)](https://www.ros.org/) and has been tested building on Ubuntu 16.04, 18.04, 20.04 systems with ROS Kinetic, Melodic, and Noetic.
1010
We also recommend installing the [catkin_tools](https://github.com/catkin/catkin_tools) build for easy ROS building.
11-
Please see the official instructions [here](http://wiki.ros.org/kinetic/Installation/Ubuntu), which have also been copied below.
11+
All ROS installs include [OpenCV](https://github.com/opencv/opencv), but if you need to build OpenCV from source ensure you build the contributed modules as we use Aruco feature extraction.
12+
See the [opencv_contrib](https://github.com/opencv/opencv_contrib) readme on how to configure your cmake command when you build the core OpenCV library.
13+
We have tested building with OpenCV 3.2, 3.3, 3.4, 4.2, and 4.5.
14+
Please see the official instructions to install ROS:
15+
16+
* [Ubuntu 16.04 ROS Kinetic](http://wiki.ros.org/kinetic/Installation/Ubuntu) (uses OpenCV 3.3)
17+
* [Ubuntu 18.04 ROS Melodic](http://wiki.ros.org/melodic/Installation/Ubuntu) (uses OpenCV 3.2)
18+
* [Ubuntu 20.04 ROS Noetic](http://wiki.ros.org/noetic/Installation/Ubuntu) (uses OpenCV 4.2)
19+
1220

1321
@m_class{m-block m-warning}
1422

1523
@par ROS Usage in OpenVINS
1624
We do support ROS-free builds, but don't recommend using this interface as we have limited support for it.
25+
You will need to ensure you have installed OpenCV and Eigen3 which are the only dependencies.
1726
If ROS is not found on the system, one can use command line options to run the simulation without any visualization.
1827
If you are using the ROS-free interface, you will need to properly construct the @ref ov_msckf::VioManagerOptions struct with proper information and feed inertial and image data into the correct functions.
1928

20-
2129
@code{.shell-session}
2230
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
2331
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
2432
sudo apt-get update
25-
sudo apt-get install ros-kinetic-desktop-full
26-
sudo apt-get install python-catkin-tools
27-
echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc
33+
export ROS_DISTRO=melodic # kinetic=16.04, melodic=18.04, noetic=20.04
34+
sudo apt-get install ros-$ROS_DISTRO-desktop-full
35+
echo "source /opt/ros/$ROS_DISTRO/setup.bash" >> ~/.bashrc
2836
source ~/.bashrc
37+
sudo apt-get install libeigen3-dev python-catkin-tools # ubuntu 16.04 or 18.04
38+
sudo apt-get install libeigen3-dev python3-catkin-tools python3-osrf-pycommon # ubuntu 20.04
2939
@endcode
3040

3141

42+
@section gs-install-opencv OpenCV Dependency (from source)
3243

33-
@section gs-install-opencv OpenCV Dependency
34-
35-
We leverage [OpenCV 3.4.6](https://opencv.org/) for this project and recommend you compile it from source.
36-
You can try linking to the one included with ROS but we have always had success with building from source to ensure we have all contributed OpenCV libraries.
37-
One should make sure you can see some of the "contrib" (e.g. xfeature2d) to ensure you have linked to the contrib modules.
44+
We leverage [OpenCV](https://opencv.org/) for this project which you can typically use the install from ROS.
45+
If these do not work (or are using non-ROS building), then you can try building OpenCV from source ensuring you include the contrib modules.
46+
One should make sure you can see some of the "contrib" (e.g. aruco) when you cmake to ensure you have linked to the contrib modules.
3847

3948
@code{.shell-session}
40-
git clone --branch 3.4.6 https://github.com/opencv/opencv/
41-
git clone --branch 3.4.6 https://github.com/opencv/opencv_contrib/
49+
git clone https://github.com/opencv/opencv/
50+
git clone https://github.com/opencv/opencv_contrib/
4251
mkdir opencv/build/
4352
cd opencv/build/
4453
cmake -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules ..
@@ -48,16 +57,15 @@ sudo make install
4857

4958

5059

51-
5260
@section gs-install-openvins Cloning the OpenVINS Project
5361

54-
55-
Now that we have ROS and OpenCV install we can setup a catkin workspace and build the project!
62+
Now that we have ROS installed we can setup a catkin workspace and build the project!
5663
If you did not install the catkin_tools build system, you should be able to build using the standard `catkin_make` command that is included with ROS.
5764
If you run into any problems please google search the issue first and if you are unable to find a solution please open an issue on our github page.
5865
After the build is successful please following the @ref gs-tutorial guide on getting a dataset and running the system.
5966

6067
@code{.shell-session}
68+
sudo apt-get install python-catkin-tools
6169
mkdir -p ~/workspace/catkin_ws_ov/src/
6270
cd ~/workspace/catkin_ws_ov/src/
6371
git clone https://github.com/rpng/open_vins/
@@ -83,4 +91,4 @@ catkin build -DDISABLE_MATPLOTLIB=ON # build without viz
8391

8492

8593

86-
*/
94+
*/

‎docs/gs-tutorial.dox

+2
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ One can take a look in the [launch](https://github.com/rpng/open_vins/tree/maste
3333
<param name="topic_imu" type="string" value="/imu0" />
3434
<param name="topic_camera0" type="string" value="/cam0/image_raw" />
3535
<param name="topic_camera1" type="string" value="/cam1/image_raw" />
36+
<rosparam param="stereo_pairs">[0,1]</rosparam>
3637

3738
<!-- bag parameters -->
3839
<param name="path_bag" type="string" value="/<path>/V1_01_easy.bag" />
@@ -111,6 +112,7 @@ Please take a look at the other launch files or the ov_msckf::VioManager constru
111112
| Group | Description |
112113
|---|---|
113114
| bag topics | ROS topics that we will parse the IMU and camera data from. If we are only using one camera, i.e. monocular, then only the first camera topic is used. |
115+
| stereo pairs | Even set of camera ids, which stereo tracking will be tried to be performed on. For example a 0,1,2,3 means that you want to perform stereo tracking of the 0,1 and 2,3 camera pairs. |
114116
| bag params. | Location of the bag we will read along with the start time, in seconds, and duration we want to run on. |
115117
| world/filter params. | This has most of the core parameters that can be tuned to improve filter performance including the sliding window size, representation, gravity, and number of environmental SLAM features. One can also change the number of cameras from 1 to 2 to do stereo matching and update. |
116118
| tracker/extractor params. | For our visual front-end tracker we have a few key parameters that we can tune, most importantly is the number of features extracted. |

‎docs/img/example_callgrind.png

186 KB
Loading

0 commit comments

Comments
 (0)