my profile | my password | search | faq & rules | forum home

» Film-Tech Forum ARCHIVE   » Operations   » Digital Cinema Forum   » GDC slow ingest (Page 1)

This topic comprises 2 pages: 1  2

Author Topic: GDC slow ingest Richard May Phenomenal Film Handler Posts: 1057 From: Floral Park, NY USA Registered: Aug 2004 posted 06-04-2017 08:22 PM                     I know it's been discussed here before but we have GDC SX-3000 IMB's with 4GB Enterprise storage. Ingest is slow as hell. Obviously, USB is going to be the slowest. Most of our content is delivered on Passport drive with USB 3.0 so that doesn't help. To make things worse, a lot of our content is 4K. The other day, I ingested a 4K movie from a passport drive into a server running a 4K movie. It took 13.5 hours. I emailed GDC about it and I was a little surprised at their response. We have only 10-15 minute intermissions and we run from 10AM till midnight everyday so there is no down time to ingest. Here is what I was told....... Just wanted to chime in here for a bit. It is highly recommended to not ingest content of any sort during playback as it can cause an issue as you are experiencing and among many others. Please do your best not to ingest content while in playback as you are limiting the thresh-hold of the drives (RAID) to a very minimum when adding data to the RAID while at the same time it is outputting data during playback. I would suggest trying your ingestion when the server is 100% idle and let us know if it continues to have the same issue and we can continue to investigate.  |  IP: Logged Frank Cox Film God Posts: 2234 From: Melville Saskatchewan Canada Registered: Apr 2011 posted 06-04-2017 09:59 PM                        For better or worse, copying data to or from a hard drive is a cpu-intensive process. (It's my understanding that SCSI drives don't require the same amount of cpu attention but cinema servers don't use scsi drives so that's no help.) I suspect that the niceness scale for an ingest while a movie is playing is set somewhere around 19 and really that's where you want it so your movie won't stutter when playing it back. In short, it's a limitation of the type of hardware in use and the only way around it would be to ingest overnight after your theatre is closed. Since it takes "all day" anyway, you might want to run it all night instead. Start the ingest before you start the night's clean-up, and when you're done that the ingest will probably be finished too. Then you can start ingesting another before you go home and just leave it running until the following morning. So that will get you two in a night.  |  IP: Logged David Stambaugh Film God Posts: 4021 From: Eugene, Oregon Registered: Jan 2002 posted 06-04-2017 10:00 PM                        In "regular" file servers, even with RAID-5, they can slow to a crawl when multiple disk-intensive processes are running simultaneously. Like if a backup is running while a user is trying to do a software build. As the IT admin I had to be very careful to schedule backups outside of work hours. What you're describing seems very similar.  |  IP: Logged Richard May Phenomenal Film Handler Posts: 1057 From: Floral Park, NY USA Registered: Aug 2004 posted 06-04-2017 10:03 PM                     Yeah we do ingest overnight sometimes. One of our screens is a rep house though so we ingest 2 to 3 movies a day in that house alone.  |  IP: Logged Steve Guttag We forgot the crackers Gromit!!! Posts: 12814 From: Annapolis, MD Registered: Dec 1999 posted 06-05-2017 07:18 AM                     The only server I know (well knew) that could multitask was Dolby's. The DSS200 can ingest, play and even export all at full speed. They didn't recommend it, but it would/will do it. I've never had a glitch when it is doing two things (play and ingesting from any source). GDC effectively doesn't ingest while it is showing a movie. It might tick up a percent or two while a movie is running and that is it. Note, you can schedule when to begin an ingest (either via the TMS or on the UI directly). An advantage of the TMS is you can ingest while shows are going and let it parse them out while the server is idle. Furthermore, FTP transfers are the fastest (and you can have a TMS push the same content to multiples).  |  IP: Logged Mark Gulbrandsen Resident Trollmaster Posts: 16657 From: Music City Registered: Jun 99 posted 06-05-2017 11:18 AM                     On all GDC servers playback takes precedence over all other functions. Even if your content was on a hard drive and you had a CRU eSATA bay, it wouldn't ingest any faster during playback of content. That's why GDC makes the enterprise server system for the SX-3000. It is able to ingest and playback at the same time. In your case that probably should have been in the spec for that system knowing how many hours a day you playback. It can always be added at any time. Mark  |  IP: Logged Monte L Fullmer Film God Posts: 8367 From: Nampa, Idaho, USA Registered: Nov 2004 posted 06-07-2017 05:54 PM                     Interesting is that when I have to do USB Sled ingests in a SX3000 Server, where we have a GDC LMS/TMS setup and I have network issues to that one house, those USB transfers can be done within 45min while the server is at rest, and I'm doing a normal sized content ingest from the host drive. Something is wacky when it takes 13.5hrs to ingest content. What about FTP transfers from server to server with both servers at rest -same speed issues?  |  IP: Logged Mark Gulbrandsen Resident Trollmaster Posts: 16657 From: Music City Registered: Jun 99 posted 06-07-2017 06:05 PM                     Monte... GDC told me that the USB speed on that mommie board is actually around a USB 2.5... So it's considerably faster than USB 2.0 would be. I have several customers that still ingest by USB and don't seem to mind it. There is also now a USB/eSATA version of the CRU's available so one can ingest via eSATA as well. How can your TMS network link be down? I never had that happen. Except for a few drive and power supply failures the TMS's have been very reliable. I've never had any issues with the network cabling. Mark  |  IP: Logged Richard May Phenomenal Film Handler Posts: 1057 From: Floral Park, NY USA Registered: Aug 2004 posted 06-07-2017 08:03 PM                     When the servers are at rest, it's fine. The problem is, they are rarely at rest. We're talking 10 minute intermissions in all 4 screens here, sometimes less. It took 13.5 hours because that day it was running 4K content all day with 10 minute intermissions.  |  IP: Logged Mark Gulbrandsen Resident Trollmaster Posts: 16657 From: Music City Registered: Jun 99 posted 06-07-2017 08:18 PM                     You have to schedule your ingests at off hours, and if there are no off hours then investing in a GDC Enterprise Storage server might be a good idea. The SX-3000 is based on a single core Atom processor, and you can only ask it to do so much. Mark  |  IP: Logged Monte L Fullmer Film God Posts: 8367 From: Nampa, Idaho, USA Registered: Nov 2004 posted 06-08-2017 03:12 PM                     Doremi has the same attribute when you try to ingest, either by CRU or USB, and you have content running, plan on a few hours for the ingest to complete. Once they get to rest, content then flies from source to target. Barco ICMB Alchemy, and Dolby aren't affected when content is being ingested while in play mode.  |  IP: Logged Mark Gulbrandsen Resident Trollmaster Posts: 16657 From: Music City Registered: Jun 99 posted 06-08-2017 03:47 PM                     Dolbly's only works going both ways because they used a top notch raid card that is able to maintain throughput at a very high rate. The high cost of building their servers is also a lot of what killed it off, er cut their profits too much. Mark  |  IP: Logged Carsten Kurz Film God Posts: 4340 From: Cologne, NRW, Germany Registered: Aug 2009 posted 06-08-2017 04:07 PM                     From my experience, Doremis will not switch back to full ingest speed if at any time after ingest started the server went into play mode. You would first have to cancel the ongoing ingest and restart to get back to full speed. Not mentioning the dreadful slow export speed...always better to pull content UUID wise through FTP. - Carsten  |  IP: Logged Bajsic Bojan Expert Film Handler Posts: 190 From: Ljubljana, Si, Eu Registered: Aug 2008 posted 06-08-2017 07:08 PM                     Don't know about that, our DCP2000 and Showvault always switch back to regular ingest speed the second a playlist stops playing, is paused or ejected. The ingest speed slows to about 5MB/s while playing back (decrease of 84% compared to USB ingest, or about 95% compared to gigabit network). The indexing is less affected (maybe runs at half the speed). Another point to mention, 4k content is rarely done at greater bandwidth and from what i've seen here, still stays at the same level as 2k content (so around 125 Mbit/s). All in all, a 14 hour ingest while playing back 90% of the time is very much expected for a lets say 180GB DCP.  |  IP: Logged Carsten Kurz Film God Posts: 4340 From: Cologne, NRW, Germany Registered: Aug 2009 posted 06-08-2017 08:17 PM                     Is that with 2.6.4? I haven't seen that happen on our Doremi. - Carsten  |  IP: Logged

All times are Central (GMT -6:00)

This topic comprises 2 pages: 1  2

Printer-friendly view of this topic

Hop To: Select a Forum: Category: Operations -------------------- Digital Cinema Forum Film Handlers' Forum Large Format Forum Ground Level Feature Info, Trailer Attachments & REAL Credit Offsets Category: Community -------------------- Film-Yak Film Handlers' Movie Reviews The Afterlife Bob Maar's Joke-A-Thon Category: Classified -------------------- Equipment and Supplies Wanted/For Sale Help Wanted/Seeking Employment Category: System -------------------- Software Beta Testing Forum

The Film-Tech Forums are designed for various members related to the cinema industry to express their opinions, viewpoints and testimonials on various products, services and events based upon speculation, personal knowledge and factual information through use, therefore all views represented here allow no liability upon the publishers of this web site and the owners of said views assume no liability for any ill will resulting from these postings. The posts made here are for educational as well as entertainment purposes and as such anyone viewing this portion of the website must accept these views as statements of the author of that opinion and agrees to release the authors from any and all liability.