How to use image subscriber?

[BRNKR]

Hey,

the tracking is working fine. But i want to fuse an image into the HMD. When I comment out #define USE_IMAGE, the node crashes. Does anyone know a current setup under Ubuntu 18.04 that works?

Ubuntu 18.04
VR Server (v1600466902)
SteamVR 1.14.16
ROS Melodic

Terminal output:
term_out.txt

[ishiguroJSK]

Can you see the steamvr home or some game images through the HMD?

If so, how about running without any image subscription?

I think, this line looks fine, but any other problems occur in the rendering process...
[ INFO] [1603726917.946537999]: Process ROS image[0] (640x480) with fov (600x600) to (1852x2056)

[ishiguroJSK]

And I'd like to confirm you are not running this node via remote terminal.
X Error of failed request: BadWindow (invalid Window parameter)

[BRNKR]

@ishiguroJSK

Thank you for your quick response and for taking the time to solve my problem.

Is it necessary to set "AllowHMD" parameter for my nvidia driver? With this Option i can put a stereocamera feed on the googles. But when I start the vive_ros node it inits a openvr app which overwrites my second screen input (3D world).

• The package works if I don't use #DEFINE USE_IMAGE
• SteamVR Home works. Therefore I assume that it is set up correctly.
• Can SteamVR be turned on? If so, do I still have to start server_vr.launch?
• I start it locally on my computer. No remote!
• I have the problem on two different computers. (RTX6000 / Titan X)
• I tried OpenVR with newest version AND the version mentioned in README
• I compiled the OpenVR Samples. HelloVR_OpenGL generates a Segmentation Error -> Vulkan sample works.

Nvidia Driver 450.80.02.

================================================================
Output server_vr.launch without manually started SteamVR:

bash: /home/gyros/.local/share/Steam/ubuntu12_32/steam-runtime/amd64/lib/x86_64-linux-gnu/libtinfo.so.5: no version information available (required by bash)
VR Server (v1600466902)

================================================================
Output server_vr.launch after start of vive.launch:

[S_API] SteamAPI_Init(): SteamAPI_IsSteamRunning() did not locate a running instance of Steam.
[S_API] SteamAPI_Init(): Loaded '/home/gyros/.local/share/Steam/linux64/steamclient.so' OK.
[S_API FAIL] SteamAPI_Init() failed; create pipe failed.Setting breakpad minidump AppID = 250820
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
[S_API] SteamAPI_Init(): SteamAPI_IsSteamRunning() did not locate a running instance of Steam.
[S_API] SteamAPI_Init(): Loaded '/home/gyros/.local/share/Steam/linux64/steamclient.so' OK.
[S_API FAIL] SteamAPI_Init() failed; create pipe failed./home/gyros/.local/share/Steam/steamapps/common/SteamVR/bin/linux64/vrmonitor: error while loading shared libraries: libQt5Multimedia.so.5: cannot open shared object file: No such file or directory
Fontconfig warning: "/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/core/panorama/fonts/conf.d/41-repl-os-win.conf", line 148: Having multiple values in isn't supported and may not work as expected
Fontconfig warning: "/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/core/panorama/fonts/conf.d/41-repl-os-win.conf", line 160: Having multiple values in isn't supported and may not work as expected
[S_API FAIL] Tried to access Steam interface STEAMREMOTESTORAGE_INTERFACE_VERSION014 before SteamAPI_Init succeeded.
/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/steamtours.sh: line 94: 525 Segmentation fault (core dumped) ${STEAM_RUNTIME_PREFIX} ${GAME_DEBUGGER} "${GAMEROOT}"/${GAMEEXE} "$@"

================================================================

Extended LD_LIBRARY_PATH with qt folder included with steam
Output server_vr.launch after start of vive.launch:

[S_API] SteamAPI_Init(): SteamAPI_IsSteamRunning() did not locate a running instance of Steam.
[S_API] SteamAPI_Init(): Loaded '/home/gyros/.local/share/Steam/linux64/steamclient.so' OK.
[S_API FAIL] SteamAPI_Init() failed; create pipe failed.Setting breakpad minidump AppID = 250820
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
[S_API] SteamAPI_Init(): SteamAPI_IsSteamRunning() did not locate a running instance of Steam.
[S_API] SteamAPI_Init(): Loaded '/home/gyros/.local/share/Steam/linux64/steamclient.so' OK.
**[S_API FAIL] SteamAPI_Init() failed; create pipe failed.**Fontconfig warning: "/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/core/panorama/fonts/conf.d/41-repl-os-win.conf", line 148: Having multiple values in isn't supported and may not work as expected
Fontconfig warning: "/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/core/panorama/fonts/conf.d/41-repl-os-win.conf", line 160: Having multiple values in isn't supported and may not work as expected
[S_API FAIL] Tried to access Steam interface STEAMREMOTESTORAGE_INTERFACE_VERSION014 before SteamAPI_Init succeeded.
/home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/steamvr_environments/game/steamtours.sh: line 94: 2228 Segmentation fault (core dumped) ${STEAM_RUNTIME_PREFIX} ${GAME_DEBUGGER} "${GAMEROOT}"/${GAMEEXE} "$@"

Output vive.launch (stripped)

++ cd /home/gyros/.local/share/Steam/steamapps/common/SteamVR/../runtime
/home/gyros/.local/share/Steam/steamapps/common/SteamVR/bin/vrenv.sh: line 5: cd: /home/gyros/.local/share/Steam/steamapps/common/SteamVR/../runtime: No such file or directory

++ cd /home/gyros/.local/share/Steam/steamapps/common/SteamVR/../sdk
/home/gyros/.local/share/Steam/steamapps/common/SteamVR/bin/vrenv.sh: line 6: cd: /home/gyros/.local/share/Steam/steamapps/common/SteamVR/../sdk: No such file or directory

libpng warning: iCCP: known incorrect sRGB profile
CQVRMonitorHeadsetWarning(0x1d11740) does not have a property named "green_color"
CQVRMonitorHeadsetWarning(0x1d11740) does not have a property named "red_color"
Initialize() is failed!!!
sh: 1: /home/gyros/.local/share/Steam/steamapps/common/SteamVR/tools/lighthouse/bin/linux64/vivetools-bv: not found

Using breakpad crash handler
Setting breakpad minidump AppID = 250820
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
Using breakpad crash handler
Setting breakpad minidump AppID = 250820
Forcing breakpad minidump interfaces to load
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
REQUIRED process [vive_node-1] has died!
process has died [pid 1270, exit code -11, cmd /home/gyros/catkin_ws/src/vive_ros/scripts/find_steam_runtime.sh /home/gyros/catkin_ws/devel/lib/vive_ros/vive_node __name:=vive_node __log:=/home/gyros/.ros/log/78c4e844-1831-11eb-9808-a4bb6dd03c72/vive_node-1.log].
log file: /home/gyros/.ros/log/78c4e844-1831-11eb-9808-a4bb6dd03c72/vive_node-1*.log
Initiating shutdown!

=============================================================

compiled with vulkan

same problems plus following:

Error opening SPIR-V file:
Error opening SPIR-V file:

• Unable to initialize Vulkan!
• Unable to initialize Vulkan!

[ishiguroJSK]

How about just launching vive.launch without server_vr.launch?
I think the SteamVR will be automatically launched.
If the SteamVR is already running, may be you have to kill the process.

[ishiguroJSK]

In the server_vr.launch, the steam related paths are set manually.

vive_ros/launch/server_vr.launch

Line 7 in 0b55e8f

<env name="LD_LIBRARY_PATH" value="$(env LD_LIBRARY_PATH):$(env HOME)/.local/share/Steam/ubuntu12_32/steam-runtime/amd64/lib/x86_64-linux-gnu:$(env HOME)/.steam/steam/steamapps/common/SteamVR/bin/linux64:$(env HOME)/.steam/steam/steamapps/common/SteamVR/drivers/lighthouse/bin/linux64"/>

On the other hand, the vive.launch uses official path setting script, and I think this is more recommended way.

vive_ros/launch/vive.launch

Line 7 in 0b55e8f

<node name="vive_node" pkg="vive_ros" type="vive_node" launch-prefix="$(find vive_ros)/scripts/find_steam_runtime.sh" output="screen" required="true"/>

find_steam_runtime.sh finds the official path setup script (because there are 2 pattern of the installation path of the script)

vive_ros/scripts/find_steam_runtime.sh

Lines 3 to 4 in 0b55e8f

	candidate_1=${HOME}/.steam/ubuntu12_32/steam-runtime/run.sh
	candidate_2=${HOME}/.steam/steam/ubuntu12_32/steam-runtime/run.sh

I hope this will help you...

[BRNKR]

Nope. Not working.

Can you tell me your setup constellation?

• Ubuntu Version
• Nvidia Driver
• OpenVR Version
• SteamVR Version (how can i download older versions?)
• any extra LD_LIBRARY_PATH in your .bashrc?

[ishiguroJSK]

I can't log in my lab PC now, but I remember the Ubuntu was 16.04, and use OpenGL (not Vulkan), and OpenVR and SteamVR were the latest ones at the time, and without any extra LD_LIBRARY_PATH.

@knorth55 @haraduka Have you installed vive_ros in 18.04?

[knorth55]

I run vive_ros with Kinetic and Melodic.

• Ubuntu 16.04 + ROS Kinetic
  • NVidia driver: 396.37
  • OpenVR: 1.3.22
  • Steam VR: 1.6.10
• Ubuntu 18.04 + ROS Melodic
  • NVidia driver: 390.116
  • OpenVR: 1.3.22
  • Steam VR: 1.6.10

[BRNKR]

You guys rock. I will try both setups.
@knorth55
Do you run it with image subscriber?

EDIT:
i downloaded steamvr 1.6.10 with https://github.com/SteamRE/DepotDownloader because it is not possible to download older steamvr versions via steamcmd anymore.

i can not use nvidia driver 390 because i have a rtx6000 which is not supported with this old driver :(
still crashes!
new error:

/home/gyros/.local/share/Steam/steamapps/common/SteamVR/bin/linux64/vrdashboard: symbol lookup error: /home/gyros/.local/share/Steam/steamapps/common/SteamVR/bin/linux64/libv8.so: undefined symbol: _ZTVN6icu_5213UnicodeStringE

EDIT2:

The RTX6000 makes problems with the new driver >400 . With an Titan Xp and Driver 390 it is not crashing!

i am getting the small preview window with my published image topics, but it doesnt render anything inside of my HMD. Do i have to use the "AllowHMD" argument for my nvidia driver or not? it is activated now and i generate an own opencv window on the extended display (HMD) with an image for each eye. what a ghettofix

[knorth55]

@knorth55
Do you run it with image subscriber?

Yes, ofcourse, and it works.

I found out that this package is pretty sensitive to nvidia driver version, so I have no idea if you can run this program with RTX6000.
with Nvidia driver 390, you can try it, I hope.
have you tried steamvr program from steam with your vive HMD?
If you can get HMD display with normal steamvr program, everything should work fine.

[BRNKR]

@knorth55
my screen on HMD is baby blue when i start the node. HMD works with SteamVR Home for example. i tried current nvidia-driver-390.138 from apt-repo with an Titan Xp. Maybe the problem is, that the System has three Titan Xp installed.