Skip to content

Latest commit

 

History

History
609 lines (450 loc) · 21.3 KB

v1.8_to_v1.9.md

File metadata and controls

609 lines (450 loc) · 21.3 KB

How to upgrade from TDW v1.8 to v1.9

Read the changelog

Read this for a complete list of changes. TDW v1.9 introduces many changes; this document only covers cases in which you might need to adjust existing code.

1. Complete re-write of the documentation

All non-API documentation has been completely rewritten. Documentation is now divided into "lessons" for specified subjects such as robotics or visual perception. You can find the complete table of contents on the README. Even if you are an experienced TDW user, we recommend you read our new documentation. You might learn new techniques!

2. Added: Add-ons

Add-ons are a new feature to TDW that will automatically inject commands into every communicate([]) call. They are very useful for commonplace tasks such as image capture as well as managing agent code.

Add-ons don't replace any existing functionality in TDW; they merely reorganize it. We recommend upgrading your code to use add-ons, but add-ons are never a necessary aspect of TDW.

3. Changes to Controller

A. Removed Controller.start()

This function used to send a command to initialize a scene in TDW. Now, that command is sent automatically in the Controller constructor.

In v1.8:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
c.start()
c.communicate(TDWUtils.create_empty_room(12, 12))

In v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
c.communicate(TDWUtils.create_empty_room(12, 12))

B. Removed Controller.load_streamed_scene(scene)

This function hasn't been the preferred way to load a stream scene for a while now because it doesn't let you send additional commands on the same frame. We recommend using Controller.get_add_scene(scene_name) instead.

In v1.8:

from tdw.controller import Controller

c = Controller()
c.load_streamed_scene(scene="tdw_room")

In v1.9:

from tdw.controller import Controller

c = Controller()
c.communicate(c.get_add_scene(scene_name="tdw_room"))

C. Removed Controller.add_object(model_name)

This function hasn't been the preferred way to add an object for a while now because it doesn't let you send additional commands on the same frame. We recommend using Controller.get_add_object(model_name) instead.

In v1.8:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
c.start()
c.communicate(TDWUtils.create_empty_room(12, 12))
object_id = c.add_object(model_name="iron_box")

In v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
c.communicate([TDWUtils.create_empty_room(12, 12),
               c.get_add_object(model_name="iron_box",
                                object_id=c.get_unique_id())])

D. Removed all librarian fields and replaced them with class variables

This allows cached librarians to be accessible outside of the controller.

self.model_librarian has been replaced with Controller.MODEL_LIBRARIANS, a dictionary that is automatically populated as libraries are added. Likewise, self.scene_librarian has been replaced with Controller.SCENE_LIBRARIANS and so on.

In v1.8:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
c.start()
c.communicate([TDWUtils.create_empty_room(12, 12),
               c.get_add_object(model_name="iron_box",
                                library="models_core.json",
                                object_id=c.get_unique_id())])
print(c.model_librarian.library) # Path to models_core.json

In v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
print(Controller.MODEL_LIBRARIANS.keys()) # dict_keys([])
c.communicate([TDWUtils.create_empty_room(12, 12),
               c.get_add_object(model_name="iron_box",
                                library="models_core.json",
                                object_id=c.get_unique_id())])
print(Controller.MODEL_LIBRARIANS.keys()) # dict_keys(['models_core.json'])

E. Removed check_build_process in the Controller constructor

This was introduced in v1.8 and ended up not being that useful; it also slows down the controller too much to be practical.

4. Refactored PyImpact

PyImpact is now an add-on and now has scrape sounds. The code for generating audio sounds is mostly the same but the code for adding PyImpact to a controller is very different, and a lot easier to use.

This is the code to required to generate impact audio in v1.8:

from pathlib import Path
from tdw.controller import Controller
from tdw.py_impact import PyImpact, AudioMaterial
from tdw.tdw_utils import TDWUtils, AudioUtils
from tdw.object_init_data import AudioInitData
from tdw.output_data import OutputData, Rigidbodies, AudioSources

c = Controller()
c.start()
commands = [{"$type": "load_scene",
             "scene_name": "ProcGenScene"},
            TDWUtils.create_empty_room(12, 12)]
a = AudioInitData(name="vase_02",
                  position={"x": 0, "y": 2, "z": 0})
object_id, object_commands = a.get_commands()
commands.extend(object_commands)
avatar_id = "a"
commands.extend(TDWUtils.create_avatar(position={"x": 0, "y": 2, "z": 2},
                                       look_at={"x": 0, "y": 0, "z": 0},
                                       avatar_id=avatar_id))
commands.extend([{"$type": "add_audio_sensor",
                  "avatar_id": avatar_id},
                 {"$type": "send_rigidbodies",
                  "frequency": "always"},
                 {"$type": "send_collisions",
                  "enter": True,
                  "stay": True,
                  "exit": True,
                  "collision_types": ["obj", "env"]},
                 {"$type": "send_audio_sources",
                  "frequency": "always"}])
resp = c.communicate(commands)
py_impact = PyImpact()
py_impact.set_default_audio_info(object_names={object_id: "vase_02"})
output_directory = Path.home().joinpath("tdw_example_controller_output/py_impact")
if not output_directory.exists():
    output_directory.mkdir(parents=True)
AudioUtils.start(output_path=output_directory.joinpath("audio.wav"))
num_frames = 0
done = False
while not done and num_frames < 1000:
    for i in range(len(resp) - 1):
        r_id = OutputData.get_data_type_id(resp[i])
        if r_id == "rigi":
            rigi = Rigidbodies(resp[i])
            for j in range(rigi.get_num()):
                if rigi.get_id(j) == object_id:
                    if rigi.get_sleeping(j):
                        done = True
                        break
    commands = py_impact.get_audio_commands(resp=resp,
                                            floor=AudioMaterial.wood_medium,
                                            wall=AudioMaterial.wood_medium)
    resp = c.communicate(commands)
audio_playing = True
while audio_playing:
    for i in range(len(resp) - 1):
        r_id = OutputData.get_data_type_id(resp[i])
        if r_id == "audi":
            audi = AudioSources(resp[i])
            for j in range(audi.get_num()):
                if audi.get_object_id(j) == object_id:
                    if not audi.get_is_playing(j):
                        audio_playing = False
                        break
    resp = c.communicate([])
AudioUtils.stop()
c.communicate({"$type": "terminate"})

This will record the exact same scenario in v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.add_ons.py_impact import PyImpact
from tdw.add_ons.audio_initializer import AudioInitializer
from tdw.add_ons.physics_audio_recorder import PhysicsAudioRecorder
from tdw.backend.paths import EXAMPLE_CONTROLLER_OUTPUT_PATH

c = Controller()
py_impact = PyImpact()
audio = AudioInitializer(avatar_id="a")
recorder = PhysicsAudioRecorder(max_frames=1000)
c.add_ons.extend([audio, py_impact, recorder])
commands = [TDWUtils.create_empty_room(12, 12)]
commands.extend(TDWUtils.create_avatar(position={"x": 0, "y": 2, "z": 2},
                                       look_at={"x": 0, "y": 0, "z": 0},
                                       avatar_id="a"))
commands.extend(c.get_add_physics_object(model_name="vase_02",
                                         object_id=0,
                                         position={"x": 0, "y": 2, "z": 0}))
recorder.start(path=EXAMPLE_CONTROLLER_OUTPUT_PATH.joinpath("py_impact/audio.wav"))
c.communicate(commands)
while recorder.recording:
    c.communicate([])
c.communicate({"$type": "terminate"})

5. Other changes to the tdw module

A. Replaced FloorplanController with Floorplan add-on

The Floorplan add-on is much more versatile; it can be appended to any controller.

FloorplanController in v1.8:

from tdw.floorplan_controller import FloorplanController

c = FloorplanController()
c.communicate(c.get_scene_init_commands(scene="1a", layout=0))

Floorplan in v1.9:

from tdw.controller import Controller
from tdw.add_ons.floorplan import Floorplan

c = Controller()
f = Floorplan()
f.init_scene(scene="1a", layout=0)
c.add_ons.append(f)
c.communicate([])

B. Replaced DebugController with Logger add-on

The Logger add-on is much more versatile; it can be appended to any controller.

DebugController in v1.8:

from tdw.debug_controller import DebugController
c = DebugController()
c.start()

Logger in v1.9:

from tdw.controller import Controller
from tdw.add_ons.logger import Logger

c = Controller()
c.add_ons.append(Logger(record=True, path="out"))
c.communicate([])

C. Replaced KeyboardController with Keyboard add-on

The Keyboard add-on is much more versatile; it can be appended to any controller.

KeyboardController in v1.8:

from tdw.keyboard_controller import KeyboardController

def stop():
    global done
    done = True
    c.communicate({"$type": "terminate"})

done = False
c = KeyboardController()
c.start()

c.listen(key="esc", function=stop)

while not done:
    c.communicate([])

Keyboard in v1.9:

from tdw.controller import Controller
from tdw.add_ons.keyboard import Keyboard

def stop():
    global done
    done = True
    c.communicate({"$type": "terminate"})
    
done = False
c = Controller()

keyboard = Keyboard()
keyboard.listen(key="esc", function=stop)
c.add_ons.append(keyboard)

while not done:
    c.communicate([])

E. Removed TransformInitData, RigidbodyInitData, and AudioInitData

These were tricky to use and unintuitive (because AudioInitData didn't actually load any audio data). They have been replaced with Controller.get_add_physics_object().

In v1.8:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.object_init_data import AudioInitData


c = Controller()
c.start()
commands = [TDWUtils.create_empty_room(12, 12)]
a = AudioInitData(name="rh10",
                  position={"x": 0, "y": 0, "z": 0},
                  rotation={"x": 0, "y": 0, "z": 0})
object_id, object_commands = a.get_commands()
commands.extend(object_commands)
c.communicate(commands)

In v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils

c = Controller()
object_id = c.get_unique_id()
commands = [TDWUtils.create_empty_room(12, 12)]
commands.extend(c.get_add_physics_object(model_name="rh10",
                                         position={"x": 0, "y": 0, "z": 0},
                                         rotation={"x": 0, "y": 0, "z": 0},
                                         object_id=object_id))
c.communicate(commands)

F. Other changes

  • Moved audio classes AudioMaterial, Base64Sound and Modes from tdw.py_impact to tdw.physics_audio.audio_material, tdw.physics_audio.base64_sound, and tdw.physics_audio.modes
  • Renamed ObjectInfo to ObjectAudioStatic and moved it from tdw.py_impact to tdw.physics_audio.object_audio_static
  • Moved AudioUtils from tdw.tdw_utils to tdw.audio_utils
  • Moved QuaternionUtils from tdw.tdw_utils to tdw.quaternion_utils
  • Moved tdw.flex.fluid_types.FluidType to tdw.flex_data.fluid_type.FluidType
  • Removed tdw.flex.fluid_types.FluidTypes Default fluid type data is now stored in a dictionary: tdw.flex_data.fluid_type.FLUID_TYPES

6. Changes to the Command API

A. Renamed send_environments to send_scene_regions

This change was made because the idiom of multiple environments no longer meaningfully exists in TDW; there are scenes, and scenes have one or more rectangular regions.

Additionally, renamed Environments output data (ID "envi") to SceneRegions (ID "sreg")

In v1.8:

from tdw.controller import Controller
from tdw.output_data import OutputData, Environments

c = Controller()
resp = c.communicate([c.get_add_scene(scene_name="floorplan_1a"),
                      {"$type": "send_environments"}])
for i in range(len(resp) - 1):
    r_id = OutputData.get_data_type_id(resp[i])
    if r_id == "envi":
        environments = Environments(resp[i])
        for j in range(environments.get_num()):
            environment_id = environments.get_id(j)
            environment_center = environments.get_center(j)
            environment_bounds = environments.get_bounds(j)
            print(environment_id, environment_center, environment_bounds)

In v1.9:

from tdw.controller import Controller
from tdw.output_data import OutputData, SceneRegions

c = Controller()
resp = c.communicate([c.get_add_scene(scene_name="floorplan_1a"),
                      {"$type": "send_scene_regions"}])
for i in range(len(resp) - 1):
    r_id = OutputData.get_data_type_id(resp[i])
    if r_id == "sreg":
        scene_regions = SceneRegions(resp[i])
        for j in range(scene_regions.get_num()):
            region_id = scene_regions.get_id(j)
            region_center = scene_regions.get_center(j)
            region_bounds = scene_regions.get_bounds(j)
            print(region_id, region_center, region_bounds)

B. Refactored Flex API

  • Renamed create_flex_fluid_object to set_flex_fluid_actor
  • Renamed create_flex_fluid_source_actor to set_flex_source_actor
  • Removed Flex commands for moving and rotating objects; instead of using these, you should move and teleport objects before adding Flex actors:
    • rotate_flex_object_by
    • rotate_flex_object_by_quaternion
    • rotate_flex_object_to
    • teleport_and_rotate_flex_object
    • teleport_flex_object

C. Split rigidbody output data into dynamic data and static data

Previously, Rigidbodies output data included dynamic data such as velocity that can change per-frame and static data such as mass that isn't expected to change per frame.

Now, that data has been split into dynamic and static output data types. Rigidbodies contains dynamic data and can be requested with send_rigidbodies. StaticRigidbodies contains static data and can be requested with send_static_rigidbodies.

In v1.8:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.output_data import OutputData, Rigidbodies

c = Controller()
c.start()
resp = c.communicate([TDWUtils.create_empty_room(12, 12),
                      c.get_add_object(model_name="iron_box",
                                       position={"x": 0, "y": 3, "z": 0},
                                       object_id=c.get_unique_id()),
                      {"$type": "send_rigidbodies",
                       "frequency": "always"}])

for i in range(len(resp) - 1):
    r_id = OutputData.get_data_type_id(resp[i])
    if r_id == "rigi":
        rigi = Rigidbodies(resp[i])
        for j in range(rigi.get_num()):
            object_id = rigi.get_id(j)
            velocity = rigi.get_velocity(j)
            angular_velocity = rigi.get_angular_velocity(j)
            sleeping = rigi.get_sleeping(j)
            mass = rigi.get_mass(j)
            kinematic = rigi.get_kinematic(j)

In v1.9:

from tdw.controller import Controller
from tdw.tdw_utils import TDWUtils
from tdw.add_ons.py_impact import PyImpact
from tdw.add_ons.audio_initializer import AudioInitializer
from tdw.add_ons.physics_audio_recorder import PhysicsAudioRecorder
from tdw.backend.paths import EXAMPLE_CONTROLLER_OUTPUT_PATH

c = Controller()
py_impact = PyImpact()
audio = AudioInitializer(avatar_id="a")
recorder = PhysicsAudioRecorder(max_frames=1000)
c.add_ons.extend([audio, py_impact, recorder])
commands = [TDWUtils.create_empty_room(12, 12)]
commands.extend(TDWUtils.create_avatar(position={"x": 0, "y": 2, "z": 2},
                                       look_at={"x": 0, "y": 0, "z": 0},
                                       avatar_id="a"))
commands.extend(c.get_add_physics_object(model_name="vase_02",
                                         object_id=0,
                                         position={"x": 0, "y": 2, "z": 0}))
recorder.start(path=EXAMPLE_CONTROLLER_OUTPUT_PATH.joinpath("py_impact/audio.wav"))
c.communicate(commands)
while recorder.recording:
    c.communicate([])
c.communicate({"$type": "terminate"})

D. play_audio_data and play_point_source_data now have a position parameter

By default, audio sources are no longer parented to objects. The id parameter is now a unique identifier for the audio source rather than an object ID. The position parameter sets the position of the audio source.

To parent an audio source to a TDW object, send parent_audio_source_to_object.

E. Refactored painting API

"Paintings" are now "textured quads".

v1.8 v1.9
create_painting create_textured_quad
destroy_painting destroy_textured_quad
hide_painting Removed; set "show" in show_textured_quad to False
rotate_painting_by rotate_textured_quad_by
rotate_painting_to_euler_angles Removed
scale_painting scale_textured_quad
set_painting_texture set_textured_quad
show_painting show_textured_quad
teleport_painting teleport_textured_quad

F. Other changes

  • Renamed parameter env_id in set_reverb_space_simple and set_reverb_space_expert to region_id
  • Removed set_proc_gen_reflection_probe. This has been replaced with enable_reflection_probes.

7. Changes to avatars

A. Adjusted A_Simple_Body

The bodies of A_Simple_Body were centered at the position of the avatar rather than halfway above. This meant that if the avatar was created at (0, 0, 0), its body would appear halfway through the floor and then "pop" up. Now, the bodies are positioned at (0, 0.5, 0) so that the avatar's pivot is at its bottom-center (just like objects in TDW).

Additionally, the cube body of A_Simple_Body required far more torque to turn than the other bodies. Now, it has a capsule collider, allowing it to spin easier.

8. Changes to controllers in the repo

A. Removed single_object.py

single_object.py and multi_env.py have been removed from the repo and have been replaced with tdw_image_dataset, a separate repo. ImageDataset is very similar to SingleObject with two notable changes:

  1. It is better organized for users to create subclasses.
  2. It replaces IdPassGrayscale data with Occlusion data, which is overall faster and somewhat less error prone.

B. Rewrote all example controllers

All example controllers have either been rewritten or removed. There are many new examples and they are grouped by documentation lesson. See here.

9. Changes to external repos

A. Upgraded Magnebot to 2.0.0

Magnebot 2 is a rewrite of Magnebot. The Magnebot is now an add-on that can be added to any TDW controller. It can still be optionally used as a controller like in earlier versions.

B. Upgraded asset_bundle_creator to 1.4.0

To upgrade:

  1. Delete this folder if it exists: ~/asset_bundle_creator
  2. Install Unity 2020.3.24f1 via Unity Hub

The next time you create a model asset bundle, the asset_bundle_creator project will be recreated.

C. Upgraded robot_creator to 1.1.0

To upgrade:

  1. Delete this folder if it exists: ~/robot_creator
  2. Install Unity 2020.3.24f1 via Unity Hub

The next time you create a robot asset bundle, the robot_creator project will be re-cloned.

D. Upgraded tdw_physics to 0.4.0

tdw_physics is now compatible with TDW v1.9.0

To upgrade: git pull

10. Updated Unity Engine

Updated Unity Engine from 2020.2.7f1 to 2020.3.24f1. This update is very unlikely to affect simulation behavior; let us know if it does.

If you have access to the TDWBase repo, you need to use Unity 2020.3.24f1 instead of Unity 2020.2.7f1.