Poor Mans Library Server / Ingest Point

[Justin d'Entremont]

I'm looking at setting up a "Poor mans library server", a computer with 5 or 6 CRU slots in it that I can pop DCP drives into, and then (hopefully without ever having to copy the content onto said computer) ingest the movies into the screen servers over the network. Does anyone use something like this that they could share how you have it setup (OS, software etc)?

In other news, what does a real LMS / TMS cost these days?

[Steve Guttag]

Seemingly, CineDigital is a good starting point for people setting up such a server. They even will let you try it out for a week and is free for up to 2-screens. It runs on a Windows computer and doesn't require a lot of space or processing power. https://www.cinedigitalmanager.com/en/

I'm not sure your plan of using a TMS as just the ingest point (not storing local) for remote servers will be a happy one. It will likely be rather slow transfers but, in a sense any computer that can support the Linux file formats of CRU drives could "FTP" content form the drive to the server but with all of the buffering, transfer rates will be rough. Why not ingest into the TMS/LMS and let it push content later...also I've yet to find a plex that needed more than 4 ingest slots.

If you have options like DCDC, then you could have the individual server just pull the content over and skip the middle man.

The problem arises though, most servers can't ingest worth a flip while they're showing movies (the Dolby DSS servers being the notable exception) so you are almost always better off to ingest to a LMS and let it push them out when time is available between shows or after hours.

[Justin d'Entremont]

Right now I do this for trailers. We have a single USB 2.0 CRU bay hooked up to the office computer (Win 10), which is running a program called ext2 Volume Manager to mount the linux partition, and then FileZilla FTP server pointed to the drive root. The servers all have this FTP point, and it works quite well, with two exceptions:

1. It's currently limited by the speed of the USB 2.0 CRU interface
2. Every time you swap the drive, you manually have to open ext2VM, locate the linux drive without a drive letter, and tell it to assign a drive letter.

Over gigabit ethernet via FTP, ingesting from one playback server to another is actually faster then directly ingesting from the CRU DCP drive, so I would think that this would still be the limiting factor if I could have a SATA interface to the drives.

My thought was to have a tower or rack unit where I could plug the 3-4 drives of what we're playing that week, along with the TrailMix drive into one location. (Reason being, our booths aren't all connected / in the same place.)


CineDigital doesn't seem like a bad option. If the professional version is sub $1,750 this might be a wise investment for us.

[Leo Enticknap]

Over gigabit ethernet via FTP, ingesting from one playback server to another is actually faster then directly ingesting from the CRU DCP drive, so I would think that this would still be the limiting factor if I could have a SATA interface to the drives.

The limiting factor could be the NIC. If you are using one gigabit card to feed multiple servers, that could be your problem. I would suggest a 10 gig SFP+ fiber card in your computer, connected to a switch on the media LAN that has an SFP+ port. That will enable you to feed up to 10 servers at a gigabit each, assuming that your motherboard and SATA interface can support that bandwidth.

Also, I would suggest a genuine Linux-based OS (e.g. Ubuntu or CentOS, for which there are many FTP server options), rather than Windows and an add-on ext2/3 reader. A Linux-based OS can read these filesystems natively, thereby taking a potential failure point out of the system.

[Marcel Birgelen]

If you don't want to invest into 10GE, but you do have a GOOD managed switch and some proper NICs (like Intel NICs) in the machine, an alternative could be using LACP and bonding multiple NICs together. If you're using decent hardware, this works pretty well on Linux and also on FreeBSD (used by software like FreeNAS). If you're using Windows, don't try to bother with it. Bonded interfaces have been a connectivity nightmare over the past decade...

[Carsten Kurz]

Some NAS/NAS adapters allow to connect and share drives, and they support the common ext2/ext3 filesystem of distribution drives.

You are talking about 'a couple of CRU slots' though - why do you need many? The moment you use more than one at the same time, your network ingest speed will drop considerably.

Which type of servers do you use, and how many?

Some NAS also offers functions to automatically copy (any) content from external drives onto their internal RAID. From there, you could ingest it to many servers, and you would also have the content stored for later.

The USB2-CRU Bays also offer a SATA port, and most PCs can easily be equipped with an eSATA slot.

In general, you should be able to share an ingest drive through a typical windows computer just as well. Those ext2/3 drivers for windows however are known to be quirky and slow, so you might be better off with a minimal Linux setup. The necessity to mount a drive before it can be shared may go away once the drive is connected through SATA, as that is usually considered a non-removable drive. However, you may need to reboot the system if you want to connect a different CRU drive. With an explicit eSATA port, that may work better, some ports can be set between SATA and eSATA in the BIOS.

Maybe you should get someone in with a decent Linux knowledge.

If you don't want to copy content onto a local computer drive to be shared from there - what it your primary reason to use network transfers?

- Carsten

[Leo Enticknap]

You can get NICs with multiple Ethernet ports on them for the scenario that Marcel describes - for example, this one has four, meaning that if you LAG-ed them, you'd have a 4 GBPS uplink out of the computer. You'd need a managed switch that can support the LAG at that end, though.

SFP+ 10 gig fiber cards are not horrifically expensive, and generic, no name transceivers that in my experience work perfectly well (e.g. FS) can be had for around $30. What does cost serious money is a decent network switch with 10 gig SFP+ capability: this is about the cheapest one with a reasonable switching capacity that I know of.

[Caleb Williams]

If anyone is checking this out in 2025, we successfully deployed old DSS200 running Ubuntu 20.04 as content management and cru ingest devices at 6 locations. Turned out the super micro server base is amazingly solid, it was the media blocks that always failed. Anyways, we've had 7 of these happily humming along since late 2019.

[Steve Guttag]

Watch out. I've had to change quite a few Super Micro motherboards (mostly X7 but some X8) on the DSS line. Yes, the RAID cards have been solid for us too. Note, I converted a DSS200 into a DSL200. The software looks to the RAID board and based on if it is a 4-channel or 8-channel, it sets the mode between DSS or DSL. What I have not completely replicated is their special multilane SAS cable...the 8-drive one is simple enough to source, but the extra part that carries data is not and may be custom. So far, the only thing that hasn't worked is running remote updates...which, at this stage of the game...isn't an issue. I also didn't put the RAID battery on the converted unit.

If you may recall, Dolby even offered a means to convert a DSL200 into a Dolby TMS (Doremi TMS) server. So, with a suitable RAID card, one could have done that with a DSS200.

Have you tried putting a USB3 port on it? (I have not) but that is something I'd think would be highly desired nowadays...for any drives that walk in the door.

[Ryan Gallagher]

Have you tried putting a USB3 port on it? (I have not) but that is something I'd think would be highly desired nowadays...for any drives that walk in the door.

Perhaps not related, but I've slapped a 2port USB-TypeC (USB3) card into my 2012 Mac Pro at home that boots Win10, and it came right up. Results will highly depend on what that particular linux kernel supports though. This was the one I used, mostly cause it was advertised to also work on macs too.

https://www.amazon.com/dp/B0763TPXRL...fed_asin_title

Ultimately the speed results may be limited by what PCI standard is on the motherboard? Based on age i'd guess PCIe 4.0. Which is a far cry better than 3.0 which this macpro has.

[Caleb Williams]

Watch out. I've had to change quite a few Super Micro motherboards (mostly X7 but some X8) on the DSS line. Yes, the RAID cards have been solid for us too. Note, I converted a DSS200 into a DSL200. The software looks to the RAID board and based on if it is a 4-channel or 8-channel, it sets the mode between DSS or DSL. What I have not completely replicated is their special multilane SAS cable...the 8-drive one is simple enough to source, but the extra part that carries data is not and may be custom. So far, the only thing that hasn't worked is running remote updates...which, at this stage of the game...isn't an issue. I also didn't put the RAID battery on the converted unit.

If you may recall, Dolby even offered a means to convert a DSL200 into a Dolby TMS (Doremi TMS) server. So, with a suitable RAID card, one could have done that with a DSS200.

Have you tried putting a USB3 port on it? (I have not) but that is something I'd think would be highly desired nowadays...for any drives that walk in the door.

Have not tried the USB3 method, however it does have PCI express on the motherboard of a DSS200, so theoretically one could install a riser card with that capability. USB 3.0 PCI cards are farely cheap on Amazon. For any booking bringing their own content we require a reasonable window for delivery before the booking to allow us to ingest, build, and QC. We haven't run into any issues just using the built-in USB 2.0 ports. A little slower, but still works just fine. The main-line kernel support for the super-micro boards is super solid based on what I've seen.

I use it for some telemetry software we've developed in-house, but the project started with the requirement to allow a way to ingest CRU drive content to Dolby TMS/IMS/Screenwriter servers. It's set up with a basic nodejs application that handles the auto-mounting of new drives, and works for anything usb. The vsftpd server filters out some directories like "lost+found" that cause issues with Screenwriter scanning for content. We also continue to use the RAID storage for content that needs to be distributed from one location to another without using hard drives. It's quite handy to be able to pull a trailer from DCDC for example, and push it across town (or across the state) without having to use a 3rd party service, a hard drive, or even a thumb drive. I can do it all from my couch at home lol. Our goal is to build out a custom monitoring solution that can catch when there are issues with equipment prior to any missed show times. As an IT department, I want to be calling a manager and having them help me fix a problem before it effects the customer experience. Right now its the other way around. The tech exists, we just have to build some custom elements and bolt it all together. I've taken it on as a sort of pet-project since our IT director is busy with other things.

[James Gardiner]

Oh, on this topic, cinema-catcher-app also support ingesting from dynamically attached hardware devices being it USB attached device or. CRU sleds via SATA. It scans pluged in devices devices, you can click on, it tries to mount and scans for DCPs. of which if found, you can then ingest. However, it only does one at a time. If you plugged in 4 disks via CRU sleds. You can scan on at a time, select the DCP, and mark for ingest. It will then ingest them one at a time, mounting them when it needs to ingest from them. its designed to only mount when needed, and only as readonly.

[Caleb Williams]

Oh, on this topic, cinema-catcher-app also support ingesting from dynamically attached hardware devices being it USB attached device or. CRU sleds via SATA. It scans pluged in devices devices, you can click on, it tries to mount and scans for DCPs. of which if found, you can then ingest. However, it only does one at a time. If you plugged in 4 disks via CRU sleds. You can scan on at a time, select the DCP, and mark for ingest. It will then ingest them one at a time, mounting them when it needs to ingest from them. its designed to only mount when needed, and only as readonly.

I didn't know this existed. Thank you!

[Hamzah Batha]

We often screen pre-release and receive quite a few films via Aspera, MediaShuttle, etc so having a 'poor mans library server' is more of a necessity. We don't get many of these come through via LANSAT.

I have a Mac Mini set up in the booth along with a G-Raid Shuttle which is connected via Thunderbolt. The Mac Mini is running FileZilla Server and the Aspera Connect software which our Dolby IMS' can ingest from via FTP. Works a charm and is decently speedy too.

[Ryan Gallagher]

We often screen pre-release and receive quite a few films via Aspera, MediaShuttle, etc so having a 'poor mans library server' is more of a necessity. We don't get many of these come through via LANSAT.

I have a Mac Mini set up in the booth along with a G-Raid Shuttle which is connected via Thunderbolt. The Mac Mini is running FileZilla Server and the Aspera Connect software which our Dolby IMS' can ingest from via FTP. Works a charm and is decently speedy too.

Similar for us, but lacking the benefit of small raid NAS or enclosure. And IT has not given me permission to put an FTP server on anything yet. So all our aspera and other downloads just go directly to large jump drives for ingesting. Much slower than FTP but as long as you stay on top of it... doable (just). Depends how many screens you are juggling. Just two here.

Our real bottleneck, which was recently improved, is there was only one machine we had local admin on to install things like the Aspera Connect downloader... and that laptop had a 10/100 nic in it. Building has 10G fiber. Now we at least have GigE on booth windows machines, and something approximating local admin escalation for installing when needed.