Merge pull request #2890 from randaz81/yarprobotinterface_enabled_tags

yarprobotinterface with `enabled_by` and `disabled_by` xml attributes

doc/module_executables/cmd_yarprobotinterface.dox

@@ -8,12 +8,12 @@
 
 \section yarprobotinterface_intro Description
 
-The yarprobotinterface is a command line tool that is useful to launch multiple YARP device at once.
+The yarprobotinterface is a command line tool that is useful to launch multiple YARP devices at once.
 
 Its name derives from the fact that the first and main use of the yarprobotinterface was used as the
 main program to provide a network "interface", via YARP Network Server Wrappers (NWS) devices, to a robot.
 
-However, the yarprobotinterface can be used to launch YARP devices of any kind. In a sense, is an extension of the
+However, the yarprobotinterface can be used to launch YARP devices of any kind. In a sense, it is an extension of the
 yarpdev command, that instead can be used only to launch one or two devices, while yarprobotinterface can launch an
 arbitrary number of YARP devices.
 
@@ -36,6 +36,12 @@ The details of the xml format of the files loaded by yarprobotinterface are docu
 - If this option is specified, then xml file is only loaded without actually opening devices.
   This option is useful to validate if xml files are well formed.
 
+`--enable_tags (xxx yyy ... zzz)`
+- This options can be used to enable optional devices which have been marked with the in `enabled_by` attribute in the xml file. See \ref yarp_robotinterface_xml_config_files
+
+`--disable_tags (xxx yyy ... zzz)`
+- This options can be used to disable included devices which have been marked with the in `disabled_by` attribute in the xml file. See \ref yarp_robotinterface_xml_config_files
+
 \section yarprobotinterface_conf_file Configuration Files
 
 yarprobotinterface loads the xml file from the location specified in the `--config` option.

doc/module_yarprobotinterface/yarp_robotinterface_xml_config_files.dox

@@ -17,10 +17,10 @@ Here is a minimal config file, let's call it "config.xml":
 <robot name="robotinterfaceExample" portprefix="/icub" build="0" xmlns:xi="http://www.w3.org/2001/XInclude">
     <devices>
         <device name="fake_motor_device" type="fakeMotionControl">
-        	<group name="GENERAL">
+            <group name="GENERAL">
                 <!-- Number of joints of the fake_motor_device -->
-		        <param name="Joints">            3         </param>
-	        </group>
+                <param name="Joints">            3         </param>
+            </group>
         </device>
         <device name="fake_motor_nws_yarp" type="controlBoard_nws_yarp">
             <!-- See https://www.yarp.it/latest/classControlBoard__nws__yarp.html for parameter documentation -->
@@ -86,5 +86,66 @@ in the `portprefix` attribute of the `robot` element.
 
 This element still needs to be documented.
 
+\section yarp_robotinterface_xml_config_files_advanced Inclusion of multiple XML files
 
-*/
+Here is an example of .xml config file which includes other xml files, using the `xi:include` tag:
+
+\code
+
+<?xml version="1.0" encoding="UTF-8" ?>
+<!DOCTYPE robot PUBLIC "-//YARP//DTD yarprobotinterface 3.0//EN" "http://www.yarp.it/DTD/yarprobotinterfaceV3.0.dtd">
+
+<robot name="exampleV2" build="1" portprefix="cer" xmlns:xi="http://www.w3.org/2001/XInclude">
+    <devices>
+    <xi:include href="wrappers/odometry/odometry_nws_yarp.xml" disabled_by="disable_odometry" /> 
+    <xi:include href="wrappers/odometry/odometry_nws_ros.xml" enabled_by="enable_ros" disabled_by="disable_odometry" /> 
+    <xi:include href="wrappers/odometry/odometry_nws_ros2.xml" enabled_by="enable_ros2" disabled_by="disable_odometry" /> 
+    <xi:include href="hardware/odometry/odometry.xml" disabled_by="disable_odometry" />
+    <xi:include href="hardware/body/body_nws_yarp.xml" disabled_by="disable_body" />
+    <xi:include href="wrappers/body/body.xml" disabled_by="disable_body" />
+    </devices>
+</robot>
+
+\endcode
+
+Contents of the file "wrappers/odometry/odometry_nws_yarp.xml".
+
+\code
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE devices PUBLIC "-//YARP//DTD yarprobotinterface 3.0//EN" "http://www.yarp.it/DTD/yarprobotinterfaceV3.0.dtd">
+
+<device xmlns:xi="http://www.w3.org/2001/XInclude" name="cer_odometry_nws_yarp" type="odometry2D_nws_yarp">
+    <param name="odometry_port_name">  /cer/odometry </param>
+    <param name="odometer_port_name">  /cer/odometer </param>
+    <param name="velocity_port_name">  /cer/velocity </param>
+
+    <action phase="startup" level="15" type="attach">
+            <param name="device">  cer_odometry </param>
+    </action>
+    <action phase="shutdown" level="5" type="detach" />
+</device>
+\endcode
+
+It must be noticed that the included file contains just a single `device` section, that will be incorporated inside the
+`devices` block in the parent file. In this way, it is possible to better organize the xml file and eventually add/remove/replace
+ individual devices.
+Additionally each device included through a xi:include can be enabled/disabled by the the use of `enabled_by` and `disabled_by` attributes.
+By default, if no attribute is added, than the file is automatically included.
+If a `enabled_by` attribute is found, then the include line is not enabled by default and it is enabled only if yarprobotinterface
+has been executed with the option `--enable_tags (xxx)` (xxx should match the contents of the `enabled_by` attribute)
+If a `disabled_by` attribute is found, then the include line (either enabled by default or by an `enable_by` attribute ) can
+be disabled if yarprobotinterface has been executed with the specific option `--disable_tags (yyy)` (yyy should match the contetes of the `disabled_by` attribute) 
+
+Examples:
+- `yarprobotinterface` starts yarprobotinterface including the following devices: odometry_nws_yarp.xml odometry.xml body.xml body_nws_yarp.xml
+- `yarprobotinterface --disable_tags (disable_odometry)` starts yarprobotinterface including the following devices: body.xml body_nws_yarp.xml
+- `yarprobotinterface --enable_tags (enable_ros) --disable_tags (disable_body)` starts yarprobotinterface including the following devices: odometry_nws_yarp.xml odometry.xml odometry_nws_ros.xml
+- `yarprobotinterface --enable_tags (enable_ros enable_ros2) --disable_tags (disable_body)` starts yarprobotinterface including the following devices: odometry_nws_yarp.xml odometry.xml odometry_nws_ros.xml odometry_nws_ros2.xml
+- `yarprobotinterface --enable_tags (enable_all)` starts yarprobotinterface including the following devices: odometry_nws_yarp.xml odometry.xml odometry_nws_ros.xml odometry_nws_ros2.xml body.xml body_nws_yarp.xml
+- `yarprobotinterface --enable_tags (enable_all) --disable_tags (disable_body)` starts yarprobotinterface including the following devices: odometry_nws_yarp.xml odometry.xml odometry_nws_ros.xml odometry_nws_ros2.xml
+
+The latter two examples show the use of the special `enable_all` tag, which can be used to enable all optional devices (i.e. marked by a `enabled_by` attribute)
+
+N.B. The `disable_tags` is always processed after the `enable_tags` and can be used to remove previously enabled devices.
+The enabled_by/disabled_by attributes are parsed by the yarprobotinterface pre-processor, hence they can be used only in combination with a xi:include tag.
+*/

doc/release/master/yarprobotinterface_enable_tags.md

@@ -0,0 +1,7 @@
+yarprobotinterface_enable_tags {#master}
+-----------
+
+### `yarprobotinterface`
+
+* `yarprobotinterface`: is now able to parse `enabled_by` and `disabled_by` xml attributes.
+See attached yarprobotinterface Doxygen documentation.

src/libYARP_robotinterface/src/yarp/robotinterface/impl/XMLReaderFileV3.cpp

@@ -22,6 +22,7 @@
 #include <string>
 #include <tinyxml.h>
 #include <vector>
+#include <set>
 
 #define SYNTAX_ERROR(line) yError() << "Syntax error while loading" << curr_filename << "at line" << line << "."
 #define SYNTAX_WARNING(line) yWarning() << "Invalid syntax while loading" << curr_filename << "at line" << line << "."
@@ -56,6 +57,7 @@ class yarp::robotinterface::impl::XMLReaderFileV3::Private
     yarp::robotinterface::Action readActionTag(TiXmlElement* actionElem, yarp::robotinterface::XMLReaderResult& result);
     yarp::robotinterface::ActionList readActionsTag(TiXmlElement* actionsElem, yarp::robotinterface::XMLReaderResult& result);
 
+    bool CheckIfIncludeSectionIsEnabled(const std::string& href_enable_tags, const std::string& href_disable_tags);
     bool PerformInclusions(TiXmlNode* pParent, const std::string& parent_fileName, const std::string& current_path);
     void ReplaceAllStrings(std::string& str, const std::string& from, const std::string& to);
     XMLReaderFileV3* const parent;
@@ -69,6 +71,10 @@ class yarp::robotinterface::impl::XMLReaderFileV3::Private
     std::string curr_filename;
     unsigned int minorVersion;
     unsigned int majorVersion;
+    std::set<std::string> m_enabled_options_from_command_line;
+    std::set<std::string> m_disabled_options_from_command_line;
+    std::set<std::string> m_enable_set_found_in_all_xml;
+    std::set<std::string> m_disable_set_found_in_all_xml;
 };
 
 
@@ -128,16 +134,61 @@ yarp::robotinterface::XMLReaderResult yarp::robotinterface::impl::XMLReaderFileV
     current_filename = fileName.substr(fileName.find_last_of("\\/") + 1);
     log_filename = current_filename.substr(0, current_filename.find(".xml"));
     log_filename += "_preprocessor_log.xml";
+
+    std::string enable_tags_string;
+    yarp::os::Bottle* be = config.find("enable_tags").asList();
+    if (be) { enable_tags_string = be->toString();}
+
+    std::string disable_tags_string;
+    yarp::os::Bottle* bd = config.find("disable_tags").asList();
+    if (bd) { disable_tags_string = bd->toString();}
+
+    {
+        char* all_string = strdup(enable_tags_string.c_str());
+        char* token = strtok(all_string, " ");
+        while (token) {
+            m_enabled_options_from_command_line.insert(token);
+            token = strtok(NULL, " ");
+        }
+    }
+    {
+        char* all_string = strdup(disable_tags_string.c_str());
+        char* token = strtok(all_string, " ");
+        while (token) {
+            m_disabled_options_from_command_line.insert(token);
+            token = strtok(NULL, " ");
+        }
+    }
+    yInfo() << "Yarprobotinterface was started using the following enable_tags:"<< enable_tags_string;
+    yInfo() << "Yarprobotinterface was started using the following disable_tags:" << disable_tags_string;
+
     double start_time = yarp::os::Time::now();
     PerformInclusions(doc->RootElement(), current_filename, current_path);
     double end_time = yarp::os::Time::now();
+
+    std::string all_enable_att_string = "List of all enable attributes found in the include tags: ";
+    for (auto it = m_enable_set_found_in_all_xml.begin(); it != m_enable_set_found_in_all_xml.end(); it++)
+    {
+        all_enable_att_string += (" " + *it);
+    }
+    yDebug() << all_enable_att_string;
+
+    std::string all_disable_att_string = "List of all disable attributes found in the include tags: ";
+    for (auto it = m_disable_set_found_in_all_xml.begin(); it != m_disable_set_found_in_all_xml.end(); it++)
+    {
+        all_disable_att_string += (" " + *it);
+    }
+    yDebug() << all_disable_att_string;
+
     std::string full_log_withpath = current_path + std::string("\\") + log_filename;
     std::replace(full_log_withpath.begin(), full_log_withpath.end(), '\\', '/');
     yDebug() << "Preprocessor complete in: " << end_time - start_time << "s";
-    if (verbose_output) {
+    if (verbose_output)
+    {
         yDebug() << "Preprocessor output stored in: " << full_log_withpath;
         doc->SaveFile(full_log_withpath);
     }
+
     yarp::robotinterface::XMLReaderResult result = readRobotTag(doc->RootElement());
     delete doc;
 
@@ -166,6 +217,50 @@ yarp::robotinterface::XMLReaderResult yarp::robotinterface::impl::XMLReaderFileV
     return result;
 }
 
+bool yarp::robotinterface::impl::XMLReaderFileV3::Private::CheckIfIncludeSectionIsEnabled(const std::string& hrefxml_enable_tags_s, const std::string& hrefxml_disable_tags_s)
+{
+    yarp::os::Bottle hrefxml_enable_tags_b;
+    hrefxml_enable_tags_b.fromString(hrefxml_enable_tags_s);
+    yarp::os::Bottle hrefxml_disable_tags_b;
+    hrefxml_disable_tags_b.fromString(hrefxml_disable_tags_s);
+    //yDebug() << "included enable tag size:" << b_included_enable_options.size()  << " contents:" << b_included_enable_options.toString();
+    //yDebug() << "included disable tag size:" << b_included_disable_options.size() << " contents:" << b_included_disable_options.toString();
+
+    //if no `enabled_by` attribute are found in the xi::include line, then the include is enabled by default.
+    bool enabled = true;
+    size_t hrefxml_enable_tags_b_size = hrefxml_enable_tags_b.size();
+    if (hrefxml_enable_tags_b_size != 0)
+    {
+        //.otherwise, if a `enabled_by` attribute is found, then the include line is not enabled by default and it
+        // is enabled only if yarprobotinterface has been executed with the specific option --enable_tags
+        enabled = false;
+        for (size_t i = 0; i < hrefxml_enable_tags_b_size; i++)
+        {
+            std::string s = hrefxml_enable_tags_b.get(i).asString();
+            m_enable_set_found_in_all_xml.insert(s);
+            if (m_enabled_options_from_command_line.find(s) != m_enabled_options_from_command_line.end() ||
+                m_enabled_options_from_command_line.find("enable_all") != m_enabled_options_from_command_line.end())
+            {
+                enabled = true;
+            }
+        }
+    }
+    // if a `disabled_by` attribute is found, then the include line (either enabled by default or by an `enable_by` tag ) can
+    // be disabled if yarprobotinterface has been executed with the specific option --disable_tags
+    size_t hrefxml_disable_tags_b_size = hrefxml_disable_tags_b.size();
+    for (size_t i = 0; i < hrefxml_disable_tags_b_size; i++)
+    {
+        std::string s = hrefxml_disable_tags_b.get(i).asString();
+        m_disable_set_found_in_all_xml.insert(s);
+        if (m_disabled_options_from_command_line.find(s) != m_disabled_options_from_command_line.end())
+        {
+            enabled = false;
+        }
+    }
+
+    return enabled;
+}
+
 bool yarp::robotinterface::impl::XMLReaderFileV3::Private::PerformInclusions(TiXmlNode* pParent, const std::string& parent_fileName, const std::string& current_path)
 {
 loop_start: //goto label
@@ -182,36 +277,59 @@ bool yarp::robotinterface::impl::XMLReaderFileV3::Private::PerformInclusions(TiX
             return false;
         }
 
-        if (elemString == "xi:include") {
+        if (elemString == "xi:include")
+        {
             std::string href_filename;
-            std::string included_filename;
-            std::string included_path;
-            if (childElem->QueryStringAttribute("href", &href_filename) == TIXML_SUCCESS) {
-                included_path = std::string(current_path).append("\\").append(href_filename.substr(0, href_filename.find_last_of("\\/")));
-                included_filename = href_filename.substr(href_filename.find_last_of("\\/") + 1);
-                std::string full_path_file = std::string(included_path).append("\\").append(included_filename);
-                TiXmlDocument included_file;
-
-                std::replace(full_path_file.begin(), full_path_file.end(), '\\', '/');
-                if (included_file.LoadFile(full_path_file)) {
-                    PerformInclusions(included_file.RootElement(), included_filename, included_path);
-                    //included_file.RootElement()->SetAttribute("xml:base", href_filename); //not yet implemented
-                    included_file.RootElement()->RemoveAttribute("xmlns:xi");
-                    if (pParent->ReplaceChild(childElem, *included_file.FirstChildElement())) {
-                        //the replace operation invalidates the iterator, hence we need to restart the parsing of this level
-                        goto loop_start;
-                    } else {
+            if (childElem->QueryStringAttribute("href", &href_filename) == TIXML_SUCCESS)
+            {
+                std::string href_enable_tags, href_disable_tags;
+                childElem->QueryStringAttribute("enabled_by", &href_enable_tags);
+                childElem->QueryStringAttribute("disabled_by", &href_disable_tags);
+                if (CheckIfIncludeSectionIsEnabled(href_enable_tags, href_disable_tags))
+                {
+                    std::string included_path = std::string(current_path).append("\\").append(href_filename.substr(0, href_filename.find_last_of("\\/")));
+                    std::string included_filename = href_filename.substr(href_filename.find_last_of("\\/") + 1);
+                    std::string full_path_file = std::string(included_path).append("\\").append(included_filename);
+                    TiXmlDocument included_file;
+
+                    std::replace(full_path_file.begin(), full_path_file.end(), '\\', '/');
+                    if (included_file.LoadFile(full_path_file))
+                    {
+                        PerformInclusions(included_file.RootElement(), included_filename, included_path);
+                        //included_file.RootElement()->SetAttribute("xml:base", href_filename); //not yet implemented
+                        included_file.RootElement()->RemoveAttribute("xmlns:xi");
+                        if (pParent->ReplaceChild(childElem, *included_file.FirstChildElement()))
+                        {
+                            //the replace operation invalidates the iterator, hence we need to restart the parsing of this level
+                            goto loop_start;
+                        }
+                        else
+                        {
+                            //fatal error
+                            yFatal() << "Failed to include: " << included_filename << " in: " << parent_fileName;
+                            return false;
+                        }
+                    }
+                    else
+                    {
                         //fatal error
-                        yFatal() << "Failed to include: " << included_filename << " in: " << parent_fileName;
+                        yError() << included_file.ErrorDesc() << " file" << full_path_file << "included by " << parent_fileName << "at line" << childElem->Row();
+                        yFatal() << "In file:" << included_filename << " included by: " << parent_fileName << " at line: " << childElem->Row();
                         return false;
                     }
-                } else {
-                    //fatal error
-                    yError() << included_file.ErrorDesc() << " file" << full_path_file << "included by " << parent_fileName << "at line" << childElem->Row();
-                    yFatal() << "In file:" << included_filename << " included by: " << parent_fileName << " at line: " << childElem->Row();
-                    return false;
                 }
-            } else {
+                else
+                {
+                    yDebug() << "Skipping include section `" << href_filename << "` because is not enabled";
+                    if (pParent->RemoveChild(childElem))
+                    {
+                        //the remove operation invalidates the iterator, hence we need to restart the parsing of this level
+                        goto loop_start;
+                    }
+                }
+            }
+            else
+            {
                 //fatal error
                 yFatal() << "Syntax error in: " << parent_fileName << " while searching for href attribute";
                 return false;