Audio Reactivity and RealSense Integration

Hey Mitja,

I managed to get all the projects running in UE4.21 with the new LA3.1.3, apart from ‘TheDepthsOfYou’:

.

I know it’s not the camera itself as it still runs on other LA projects that require the RealSense webcam.

Please could you help? Thank you.

Love the projects by the way =]!!!

All the best,

Smudgy

Hey Smudgy,

I’m glad you like them!

I just confirmed on our side and the project works here. Having said that RealSense can be a bit fidgety when it comes to repeated connects/disconnects and it also depends on how fast your USB controller is. Sometimes even the Intel’s RealSense software (Intel RealSense Viewer) cannot connect.

So, I’m afraid I don’t have any other suggestions at this point than the lame ones:

  • restart the computer and disconnect/reconnect RealSense.
  • Start Lightact and open the The Depths of You project and see if it works.
  • If it still doesn’t work, delete the RealSense Reader node in the Devices window and create a new node, select the RealSense camera in the Stream section and try to connect it again.

I hope this helps.
Mitja

Hey Mitja,

They’re excellent, most impressed.

The SR300 works for Windows Hello, Intel RealSense Viewer, my quick check program (see attached file) and also my video capture program, named, Action!

  • I’ve restarted the computer and disconnected/reconnected the RealSense

  • Started LightAct and opened the project, the texture panel is relayed:

  • I’ve deleted the RealSense Reader node, saved, closed and reopened the project. Added the new Reader node, named RealSense SR300 and on attempting to stream, the error is still present:
    The USB port I am connecting through is a 3.1 Gen2 type.

Have you possibly any other ideas, please?

Kind regards,

Smudgy

RealSense.la (8.54 KB)

Hey Smudgy,

The first thing that I noticed is that we used SR435 during development, so I was thinking that SR300 might not be supported. But then, you are saying that the camera itself works when you use it in other sample projects? Which sample project was that? I am not aware of any that use RealSense depth feed (but then again, I might be wrong).

In short, can you please confirm that you can see the depth feed from your RealSense in Layer Layouts in a sample project (or a project built from scratch)?

Thanks!
Mitja

Hey Mitja,

Thank you, as ever, for your help.

Did the attached project not come through, please find attached now. This project uses
both the depth field and RGB streams.

The attached project yields the following resultant outputs:

Hope you can help more?

What would have I have tio do with the project to convert it for use for the SR300 camera, please?

Kind regards,

Smudgy

RealSense.la (8.54 KB)

Hi,

We are going to have to have a more in-depth look at this, I am afraid. From the initial looks of it, things get a bit sketchy when opening projects saved with different RealSense camera.

At the moment, the only thing I would recommend is for you to recreate the project from scratch.

I’ll update this thread with new findings.

Thanks,
Mitja

Thank you Mitja,

I’ll recreate once I’m home later and let you know the result.

All the best,

Smudgy

Hey Mitja,

I hope you’re well… Happy Friday!

I have rebuilt the project from scratch (please find attached) and still the issue persists. How does your project receive the data from the webcam and what possible differences are there between the SR300 and D435 in relation to the datastream inputs into LA?

I only recently purchased my BlasterX Senz 3D Webcam (Intel SR300), though if I need to get the D435/D435i (which one would you choose?), I will; I really want to get my head round this and do it all properly and if it means upgrading the webcam, then so be it.

I’m interested in purchasing a suitable LA license and have requested the associated prices for each type.

Have a great weekend!

Kind regards,

Dominic

TheDepthsOfYou313_Rebuild.la (19.1 KB)

Hey Dominic,

Happy Friday to you too!

I’ve opened your file and it works on my side with D435

I’m afraid if SR300 doesn’t work with Lightact then there’s not much we can do at this point (aside from you sending your camera over so we can integrate it and send it back :slight_smile: ) I am just curious why it seems that it works with other projects (if I understood you correctly). If yes, then at which point does it stop working? Please note that knowing this won’t really help us fix the issue without having the camera here, so if you want to be up and running asap I am afraid you’ll have to buy a new camera. Sorry about that.

We are using D435, so if you want to be 100% certain, you’ll probably want to use the same one.
IMG_20190125_145810

I know @Jason_Ellemor (let’s see if this ‘@’ thing works) has been using RealSense with Lightact successfully. Jason, which RealSense are you using if you don’t mind me asking?

Thanks!
Mitja

Thank you Mitja =].

Only 3 hours of shift left… and counting.

This is most bizarre. There’s two LA projects it works for, the one named ‘RealSense’ of I have attached already and your other example project, ‘Multiverse’, where you wave your hands in front of the webcam and the small squares on a network grid light up.

TheDepthOfYou project fails when on Devices (F7) window, when the camera RealSense SR300 #Serial Number input is selected.

When the new webcam comes, I’ll send the SR300 to you to see if you can get it working.

Oh well that was $189 dollars down the drain for the SR300. Just about to order the D435i, so I’ll give you an update when it arrives and I have it setup.

All the best and enjoy the weekend =].

1 Like

I am using Real Sense D435 without any issues. I’ve found that you can have issues if you disconnect the real sense camera during a project. Try opening the realsense depth viewer (or other tools) to make sure the camera is working, then kick off LightAct and it will generally work without an issue

1 Like

Thank you Jason for the response.

I have just ordered the D435i, so this will just work as you and LA say.

The bizarre thing is I have two other LA projects that use the SR300 and they both work fine. Also does the Intel Viewer and other apps that use the camera.

I have an up to date system and have USB3.1 ports, so when the USB connection error comes on in the LA project upon selecting the camera device input, it really puzzles me.

Hey Mitja,

I hope you’re ok and all is well?!

Ok, finally got my D435i and setting it up, calibration completed successfully.

Opened the LA project, TheDepthsOfYou and saved as TheDepthsOfYou 313.
Opened the UE4 project map, depthsOfYou and hit run, though nothing appears on planes.

On opening Devices window:

If I hit ‘Refresh List’, the camera doesn’t show.

The D435i works with the Intel RealSense Viewer V2.17.1:

Any ideas, please?

All the best,

Smudgy

Hey,

Hmm that’s weird. Just to be sure, can you try to:

  1. Restart your pc

  2. Start Lightact and open Devices window. Don’t open the Intel real viewer just yet.

  3. Insert RealSense Reader node and see if you see your RealSense in the list?

Sometimes I got somewhat similar symptoms on a relatively old PC with what I guess were older USB ports/controllers so the RealSense connection was sometimes a bit dodgy. What helped me on that particular pc was to get an additional PCIe expansion slot. I’m not saying this is the way you should go though - just stating my experience.

Let me know how it goes please.

Best regards,
Mitja

Hey M,

Tis weird too!

The camera doesn’t show up in list at all.

Other programs can use it though.

So I followed steps 1 to 3 and nada.

My machine is new with USB3.1, type C ports, the mind boggles!

Kind regards,

Smudgy

Hey M,

Tis weird too!

The camera doesn’t show up in list at all.

Other programs can use it though.

So I followed steps 1 to 3 and nada.

Please see attached machine spec.

Kind regards,

Smudgy

(Attachment Zeus_11-02-2019.nfo is missing)

Hey Mitja,

In relation to your D435 camera settings for your project, are they like this:

.

And are your drivers like this:
image

image

This is so confusing, I really would just like to develop as opposed to fix things, lol.

All the best,

Smudgy

1 Like

Hey Mitja,

Would it be possible to for you to test the D435 using the latest version of firmware (5.11.01.00) and SDK (V2.18.1), from Intel, please?

Hey Smudgy,

Yes, I’ll look into it now and let you know.

The firmware number is the same but I’ll update the SDK version now. I was using 2.17.

image

image

I’ll let you know the news once I install SDK 2.18.

Cheers,
Mitja.

Hello again,

‘Unfortunately’ it worked even with SDK 2.18. on my end.
image

image

Procedure with SDK 2.18.1

Procedure with SDK 2.17.0

It seems the only difference now between our setups is the fact that I am using D435 and you are using D435i. I looked a few posts above, and it seems Jason was also using D435. As I don’t have it here atm, its a bit hard to test, but I’ll give it a try.

If you could send it over, we’ll be happy to have a look at it though. :slight_smile:

One more thing to try: could you please install SDK 2.17.0 and give it a go?

Thanks!
Mitja