Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Slow Dahua RTSP stream with VLC when going through pfSense

    Scheduled Pinned Locked Moved
    General pfSense Questions
    rtsp stream sg-3100
    2
    8
    1.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      harefoot
      last edited by

      Hi all,

      We have a situation where an RTSP stream viewed with VLC on the VLAN is quick to start (1 second), but when it goes through pfsense and a different VLAN, it's very slow (12 seconds). This is reproducible and has been tested with multiple machines on both Windows 10 and MacOS.

      The specific setup is as follows:

      • Camera: Dahua DH-IPC-HFW4613H-ZSA (as one example, but also an Amcrest 4k which is Dahua hardware)
      • Client hardware: Dell with Intel i9 or MacBook (same results)
      • Switches: all Gigabit D-Link Web Smart Switch (a couple of models betweeen the PoE feeding the cameras and the computer, but all D-Link Web Smart)
      • Firewall: pfsense on a Netgate SG-3100
      • Client software: Win10 or MacOS; VLC 3.0.8 Vetinari

      We set substream 1 to a low resolution for testing:
      H.264H, 704x576(D1), 20FPS, VBR, Quality 6

      We start VLC with:
      vlc -vvv "rtsp://admin:xxxxxxxxxxx@192.168.200.123/cam/realmonitor?channel=1&subtype=1"

      If the machine running VLC is on VLAN200, it consistently starts in almost exactly 1 second.
      If the machine is switched to VLAN1, it consistently takes 12 seconds.

      To prove there's nothing else in the mix (e.g. VLC settings or OS firewall stuff), we have performed the following test. Take a Mac notebook with DHCP enabled and wifi disabled. Plug its Ethernet into a switch set to untagged VLAN1.

      • Refesh address via DHCP
      • get an address on 192.168.34 (which is for legacy reasons, currently VLAN1)
      • Issue the command: vlc -vvv "rtsp://admin:xxxxxxxxxxx@192.168.200.123/cam/realmonitor?channel=1&subtype=1"
      • Wait 12 seconds, and it appears.
      • Switch the port to untagged VLAN200 (removed from 1, add to 200, actually)
      • Issue the exact same command again: (vlc -vvv "rtsp://admin:xxxxxxxxxxx@192.168.200.123/cam/realmonitor?channel=1&subtype=1")
      • Wait 1 second
      • it appears

      I am new to pfsense, but I can't imagine it's this slow... We must be doing something wrong. The specific hardware (SG-3100) comes highly recommended and we didn't take any chances with setting up our own box (however tempting that was!)

      I've heard mixed reviews about whether camera streaming should all be UDP but didn't get that working after a medium amount of effort. I'm interested in best practices there and have lots of questions, but separately, I'd like to get to the bottom of why this is painfully slow because there are lots of other IoT things we were hoping to use the pfsense+VLANs to secure.

      This is our first test and it was really exciting seeing everything come together until this. Does anyone have advice?

      Note also that both the Mac notebook and new Windows 10 machines behave the same: ~1 second on VLAN1, ~12 seconds on VLAN200 with pfsense doing L3 routing. So... different OS and hardware, but same results based only on switching from VLAN200 to VLAN1+pfsense.

      Finally, and perhaps very significant--or maybe not:

      We have a middle-aged iPhone 8 on wifi(obviously?) on the 192.168.34 (i.e. VLAN1) which is, as per above, the super slow access situation. On it, we have a "Live Cams Pro" app configured with the exact same RTSP URL. It loads in maybe 2 seconds and it is also controlled by pfsense... same network as the Mac notebook and the Win10 PCs. So... the best we can come up with is that the problem is somehow tied to the interaction between VLC and the pfsense and neither one individually, since VLC can be very fast with everything the same but bypassing pfsense and being in the same VLAN... but pfsense routing the stream can be pretty fast going to the iPhone.

      We'd very much appreciate any ideas!

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Once the stream starts it is OK I assume? So an issue initiating the connection somehow?

        Run a packet capture in pfSense. Get one from each VLAN showing the slow startup. Check where the delay is.

        I very much doubt it will be across pfSense directly.

        Steve

        1 Reply Last reply Reply Quote 0
        • H
          harefoot
          last edited by

          Hey Steve... thanks for the quick reply!

          Yes, the stream looks fine once it starts.

          I ran the packet capture on the version that goes through pfsense (i.e. cam on VLAN200, PC and VLC on VLAN1). I don't think the fast version touches the pfsense because the computer is on the same VLAN ... so I think it's all being handled by the DLINK switch at L2, however I am new to this.

          The slow version packet capture, going through the pfsense, looks like this. I'm not sure if this is the right level of detail to debug. I do see a period of 10.5 seconds of no activity, bolded like this, but don't know what step to take next.

          08:34:33.366568 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 0
          08:34:33.367243 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 0
          08:34:33.367798 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 156
          08:34:33.367810 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 0
          08:34:33.368420 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 0
          08:34:33.368572 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 0
          08:34:33.377738 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 142
          08:34:33.378296 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 390
          08:34:33.400322 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 158
          08:34:33.401079 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 416
          08:34:33.438805 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 0
          08:34:33.526835 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 861
          08:34:33.537137 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 451
          08:34:33.537534 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 0
          08:34:33.545355 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 174
          08:34:33.546721 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 473
          08:34:33.566063 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 174
          08:34:33.568507 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 430
          08:34:33.586966 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 159
          08:34:33.627068 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 0
          08:34:44.119216 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 415
          08:34:44.119233 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 0
          08:34:44.122312 IP 192.168.200.123.554 > 192.168.34.141.65310: tcp 50
          08:34:44.122748 IP 192.168.34.141.65310 > 192.168.200.123.554: tcp 0
          08:34:44.123510 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 0
          08:34:44.123914 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 0
          08:34:44.124354 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 0
          08:34:44.134566 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 156
          08:34:44.135100 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 0
          08:34:44.138248 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 142
          08:34:44.138786 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 390
          08:34:44.177881 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 158
          08:34:44.178926 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 416
          08:34:44.218676 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 0
          08:34:44.325345 IP 192.168.200.123.554 > 192.168.34.141.65311: tcp 861
          08:34:44.327936 IP 192.168.34.141.65311 > 192.168.200.123.554: tcp 447

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            I assume 192.168.200.123 is the camera there?

            What interface it that capture taken on? The camera side?

            You'd have to check the packet capture itself to see what's happening there. There might be a clue as why nothing happens for 10s.

            Steve

            1 Reply Last reply Reply Quote 0
            • H
              harefoot
              last edited by

              Yes, 192.168.200.123 was the camera in that capture. I just did another similar capture to a different camera (192.168.200.122) after disabling auth on the camera, so that if it were helpful, I could send the full capture without exposing the password.

              We're currently only using the OPT1 physical interface, as we're testing and also because ultimately we expected to have more VLANs that there are physical ports on the pfsense--so we will ultimately need multiple VLANs on one port. My understanding is that at some point we probably want to do more with physical port separation for performance, etc. but that this should be "ok" for now. I'll attach a screencap of that config as well as the capture parameters.

              screencap_20200129_104651.png

              screencap_20200129_102035.png

              Finally, here's a screencap of a Wireshark view of the capture. I could include the dump but maybe this tells enough for someone who better understands this than I do...

              screencap_20200129_105406.png

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                It looks like the client is expecting the camera to send data and after 10s it tears down the connection.

                If the camera is sending anything it's not TCP on port 554 so we can't see it.

                Probably better to just capture all traffic to the client on VLAN 200.

                Also; the client sends the password unencrypted? 😕

                Steve

                1 Reply Last reply Reply Quote 0
                • H
                  harefoot
                  last edited by

                  I got caught up with other work and couldn't look at this for awhile. I'm happy to say the issue has been resolved and I wanted to update for anyone that might be reading this down the road, and also to confirm there was no problem with the pfsense hardware.

                  @stephenw10 if you're commenting on me temporarily removing the password, it's not that I actually saw the password being passed unencrypted--I just removed it in case it could be leaked somewhere and because it was easy to do. If you actually saw it in there, let me know! Thanks for the packet capture advice. That was much easier than I expected and will be helpful for other stuff. The way it works in pfsense is great, with the summary in the GUI but with the additional ability to download and view in Wireshark. Both worked really well and were surprisingly easy.

                  The issue was just the connection was being attempted as UDP, but I had not allowed those ports to the cameras. Per my original post, I wanted to use TCP and had only allowed TCP/554 to pass through. I guess(?) what was happening was that VLC was attempting UDP first and reverting to TCP when that didn't work.

                  I can make it work 3 different ways (independently; any one of these is sufficient--you don't need to do all 3):

                  • allow all UDP into the camera
                  • add the command line option --rtsp-tcp when starting VLC
                  • change preferences, Live555 stream transport to RTP over RTSP (TCP); this is in Input/Codecs, toward the bottom

                  Thanks again for the quick responses.

                  1 Reply Last reply Reply Quote 1
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    Nice catch! Thanks for the follow up. 👍

                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post