Launchbox video issue

[Jmricher]

Hi

since yesterday most video files are downloaded 0 bytes. 
 

I’ve been able to dl a few of them this morning but most of them are empty. 
 

need help please 

[Circo]

I believe pretty much all issues are fixed with the current API and Launchbox.  Speed and responsiveness seems to be holding steady at "better than ever" performance.  This should be true for all versions.

In the current LaunchBox/Bigbox beta release there is a new Launchbox integration which will have the same performance (or better) as the built in LB database as it's a scaled down and a more direct version specific to LaunchBox/BigBox. This new integration also resides on a different server to help balance the traffic more effectively.

Please let us know of any issues.

[Circo]

Site Announcement

During the last week some LaunchBox users have reported an issue with 0KB downloads and other errors when using the EmuMovies Sync integration.  After a week of digging we have finally isolated the cause of the problem.  This error has also had the side affect of affecting server processes slowing down results and in some instances caused timeouts in all API integrations including the EmuMovies Sync app.

Current ETA on the FIX is 4/11/2022

We would like to thank everyone for being patient while we work on the API. 

[GamingZone]

For the last few days when I try and add or update a game in Launchbox it seems to hang forever on the connection to Emumovies.  Others on the Launchbox forum have also seen similar issues.  Is there an issue with the API Launchbox hits, or the pipeline to the API?  Any thoughts on the issue?

I appreciate your time.

[Fearhad]

I am having the same issue. Starting a couple of days ago.

[Wanderer189]

I am also having the same problem along with errors galore.

Seems to be a "timed-out" issue for me. Getting 1000's of error the past couple days, especially with video snaps and taking forever to scrape (many minutes).

[Circo]

We are looking into this with help from Jason over at Launchbox.  Everything "seems" fine with the API and we have been unable to replicate the problem.  Unfortunately it seems to be affecting a very small percentage of users making it difficult to pinpoint the problem.  Currently we suspect it's something to do with cloudflare and are trying some things to see if it resolves to issue.

[Jmricher]

1 hour ago, Circo said:

We are looking into this with help from Jason over at Launchbox.  Everything "seems" fine with the API and we have been unable to replicate the problem.  Unfortunately it seems to be affecting a very small percentage of users making it difficult to pinpoint the problem.  Currently we suspect it's something to do with cloudflare and are trying some things to see if it resolves to issue.

In my case the download seems to operate but at the end of the operation I only have 0 byte mp4 files. 

[Jmricher]

4 hours ago, Circo said:

We are looking into this with help from Jason over at Launchbox.  Everything "seems" fine with the API and we have been unable to replicate the problem.  Unfortunately it seems to be affecting a very small percentage of users making it difficult to pinpoint the problem.  Currently we suspect it's something to do with cloudflare and are trying some things to see if it resolves to issue.

In my case the download seems to operate but at the end of the operation I only have 0 byte mp4 files. 

[Jmricher]

Works again tonight …

[Circo]

1 hour ago, Jmricher said:

Works again tonight …

We haven't actually changed anything, beyond some minor housekeeping as we look for an issue.  If it's anything outside of LB it's our CDN, they have had some global routing issues but technically it shouldn't have caused the outage.  We are continuing to collect data on the issue.  It is interesting that in your support ticket upgrading to the latest LB beta version seems to have solved it but still not convinced that is the issue.

[Circo]

We have reset our CDN cache which should hopefully alleviate any bad data.  Hopefully this resolves the issue for those few who were affected.

[Fearhad]

Still not working for me. Maybe I'll try the latest launchbox beta.

To clarify, it's not even downloading a 0 byte file. It just sits at Querying EmuMovies...

And the latest launchbox beta did not help. 

 

[Evil]

Same problem here.

I'm on the latest stable version of LaunchBox and when I try to download media from Emumovies it keeps trying to connect forever.

[Wanderer189]

This continues being an issue for me as well. Scraping takes forever, then, by the time it finishes and downloads finally start, after 20-30 minutes or so, the downloads slow to a crawl (I'm talking 20-30 seconds to download a png file) and stays that way until the LB process finishes. Then after hours of waiting, I get 1000's of errors in LB (most errors are "timed out" and mostly videos).

Kinda making the feature in LB using Emu useless for me now and a HUGE waste of time. LB team is just providing this: 

Administrators
Posted Tuesday at 12:20 PM
"While this is a confirmed issue on EmuMovies end, if you do run into EmuMovies download issues, we encourage you to open up a support ticket on their website"

This has been an issue for me for a couple weeks now, I sure hope this doesn't turn into one of them "left hand blames the right hand" issues and the users that have the PROBLEM like me are left to hang until some accidental fix comes along, if it does.

Sorry for the rant, I love both sites

[Circo]

You guys that are experiencing the issue can you download our EmuMovies Sync app and compare performance and see if the problem continues?  We are hard at work trying to figure out what the heck the issue is and any data can be incredibly helpful.

[Wanderer189]

On 4/9/2022 at 9:54 AM, Circo said:

You guys that are experiencing the issue can you download our EmuMovies Sync app and compare performance and see if the problem continues?  We are hard at work trying to figure out what the heck the issue is and any data can be incredibly helpful.

@Circo   Been a long time since I used it. Here are my results:

After starting app, took about 2 minutes to download systems. After that, I configured it to download all Sega Saturn (about 250 roms), then I waited 43 seconds before it logged in.

After it started downloading. "All Artwork" took 22 minutes. Then started HQ Videos, took 54 seconds for login and create folder, then finished in just under 18 minutes. This would have taken HOURS via LB since this stuff started (I know, should have used smaller platform to test with)

Hope this might help in some way. Maybe later today I will downgrade LB and see if still has an issue, but folks over there are saying it isn't helping them.

Thanks for the reply and anything I can do to assist, just let me know.

[Evil]

16 minutes ago, Wanderer189 said:

@Circo   Been a long time since I used it. Here are my results:

After starting app, took about 2 minutes to download systems. After that, I configured it to download all Sega Saturn (about 250 roms), then I waited 43 seconds before it logged in.

After it started downloading. "All Artwork" took 22 minutes. Then started HQ Videos, took 54 seconds for login and create folder, then finished in just under 18 minutes. This would have taken HOURS via LB (I know, should have used smaller platform to test with)

Hope this might help in some ay. Maybe later today I will downgrade LB and see if still has an issue, but folks over there are saying it isn't helping them.

Thanks for the reply and anything I can do to assist, just let me know.

I think the download times with Emumovies Sync are still very high. At least I remember that when I used this program it didn't take that long to download the entire average of 200 roms. Maybe I'm wrong, but I think there are a lot of bad connections to the server and this is why it takes so long to download the content.

[Wanderer189]

If Emu is using servers from the same company that LB was using back before the January migration this year, then that is probably the issue. I had huge problems with uploading and downloading. Once Jason moved everything I haven't had a single issue since.

[Bitstorm]

Same issue here.

Just a thought, could it be some race condition/anti-flood that affects high speed connections?

I'm on 500 Mbps fiber, saw someone mention having the same issue on gigabit fiber on LB forums.  He said it helped when he connected some VPNs (usually slowers down connection speed as a side effect).

Ftp works great, I'll try the Sync app when I get a chance.

[Wanderer189]

41 minutes ago, Bitstorm said:

Same issue here.

Just a thought, could it be some race condition/anti-flood that affects high speed connections?

I'm on 500 Mbps fiber, saw someone mention having the same issue on gigabit fiber on LB forums.  He said it helped when he connected some VPNs (usually slowers down connection speed as a side effect).

Ftp works great, I'll try the Sync app when I get a chance.

I doubt it, I only get 30-80 Mbps downloads

[Bitstorm]

On 4/9/2022 at 10:54 AM, Circo said:

You guys that are experiencing the issue can you download our EmuMovies Sync app and compare performance and see if the problem continues?  We are hard at work trying to figure out what the heck the issue is and any data can be incredibly helpful.

The Sync app does suffer the same delays.

I've added exclusions in Bitdefender: it didn't help.

After setting up Login informations, options in Download tab remain grayed out for a full minute before you can do anything (same if you close/reopen the app).

Delays aren't always exactly the same.

Status example:
Login: 15:58:24 (hh:mm:ss)
Downloading Systems: 15:59:44
(some other activity under the hood): 15:01:06
Options finally "un-grayed": 16:02:27

Then you configure options (e.g. Boxes for 1 system for less than 100 roms) and hit Go:
Systems Downloaded: 00:00
Login: 00:40
Processing Search on Server: 01:19
Downloads finally start: 01:50 (on another try I reached this point at 02:40 instead of 01:50)
Once downloads finally start, it downloaded 80 files (38 MB) in about 15 seconds.

When checking activity "under the hood", it does some operations, then pause for several seconds, do a bunch of operations real quick again, pause for several seconds, and so on.  Make sure it's not some form of automatic flood/attack protection on the server side (or CloudFlare).

I could send you a Process Monitor (Sysinternals) trace to help debug, if there's a way to send them privately.

[tailslide]

Nothing downloads for me not even 0 byte files. Getting timeout errors.

 

[Circo]

1 hour ago, Bitstorm said:

The Sync app does suffer the same delays.

I've added exclusions in Bitdefender: it didn't help.

After setting up Login informations, options in Download tab remain grayed out for a full minute before you can do anything (same if you close/reopen the app).

Delays aren't always exactly the same.

Status example:
Login: 15:58:24 (hh:mm:ss)
Downloading Systems: 15:59:44
(some other activity under the hood): 15:01:06
Options finally "un-grayed": 16:02:27

Then you configure options (e.g. Boxes for 1 system for less than 100 roms) and hit Go:
Systems Downloaded: 00:00
Login: 00:40
Processing Search on Server: 01:19
Downloads finally start: 01:50 (on another try I reached this point at 02:40 instead of 01:50)
Once downloads finally start, it downloaded 80 files (38 MB) in about 15 seconds.

When checking activity "under the hood", it does some operations, then pause for several seconds, do a bunch of operations real quick again, pause for several seconds, and so on.  Make sure it's not some form of automatic flood/attack protection on the server side (or CloudFlare).

I could send you a Process Monitor (Sysinternals) trace to help debug, if there's a way to send them privately.

Now we are talking, you can send anything privately over pm or support ticket.  Although the timings you are listing are not that abnormal for the weekend.  You request gets put in a queue and then it does a file matching operation remotely.  The actual downloads are cloud cached but the entire match and map operation happens before downloads start.  The bigger the batch the longer it takes.  

But this is some actual data I can send to the dev.  We have discussed moving some of these operations to the users local pc but it kind of nerfs the outside API integration.  

[Circo]

1 hour ago, tailslide said:

Nothing downloads for me not even 0 byte files. Getting timeout errors.

 

Any chance you are using a VPN?

[Bitstorm]

5 hours ago, Circo said:

Although the timings you are listing are not that abnormal for the weekend.  You request gets put in a queue and then it does a file matching operation remotely.  The actual downloads are cloud cached but the entire match and map operation happens before downloads start.  The bigger the batch the longer it takes.

4 mins just to open the tool, logon and download a list of available systems/art types, this is very long.  If this not considered that abnormal, your API definitely needs huge optimization.  Is your server taking that much of a beating during weekends?

Right now I'm getting much shorter times for sync app: about 50 secs startup and 50 secs for downloads (full re-download of the very same 80 files set as earlier), but the pre-process is still very long.

From LaunchBox, querying against a single rom title (Super Mario World for SNES) finds 25 artworks instantly from LBGD then queries EmuMovies for 4.5 mins before getting results (for a single rom, ouch!).

So EmuMovies Sync app does have some delays querying the API (looks like processing on the API server), but from LauchBox it's much worse.

Can you suggest some other 3rd party apps supporting EmuMovies API for comparison?

-EDIT- trace files sent, ref: Support request #9707.