Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Program Announcements / [Rewrite] Download plugin (with progress tracking)

Author
Message
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 03:23 Edited at: 30th Dec 2012 21:56
I decided to rewrite this to address the issues that seemed to occur in my old one when used with Windows 7. So far this one indeed seems to work with Win7, albeit I haven't tested it with any other OS's so if you can do that and drop me a line telling whether there are any issues I would much appreciate it

Update as of 2012-12-30
----------------------------------------
The plugin has been changed a bit in a way that breaks backwards compatibility.
This is mainly through the introduction of manual resource freeing which will have to be carried out for each download once it is finished. This gives you various perks, most importantly it removes the need for the plugin to give extra time to download threads to finish which in turn led to slowing down things when you had lots of downloads. It also allows you to properly query the "finished" status of a download (in the previous release all such functions would immediately report zeroes or "no such download exists") for as long as you see fit.

Added functions:

All of these have their own page in the documentation. Some other pages have been updated as well.
----------------------------------------

Features
♦ Asynchronous download of an arbitrary number of files.
♦ Download progress can be polled in realtime (for example number of bytes downloaded so far and total download size).
♦ Possibility to ignore cached downloads to ensure that the user always gets the latest version of a file.

Download

Verified to work on
♦ Windows 8 Professional
♦ Windows 7 Professional
♦ Windows XP SP2

Simple demo


Notes
The number of possible simultaneous downloads may vary from system to system. This seems to be bound to the number of processor cores available.


"Why do programmers get Halloween and Christmas mixed up?"
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 17th Dec 2012 04:47 Edited at: 17th Dec 2012 08:27
Thank You¡

Will test as soon as possible...

My test OSs are Win8 Pro x32/64... Win7 Ultimate x32/64... Vista Home Premium and Ultimate x32... XP SP3 x32/x64...

I will test on as many of them as I can... let me know if there are specific ones you particularly would like tested

Again... Thank You!

EDIT

TOTALLY WORKING!!!!!!!

You are all welcome to use a file from my server to test with and play it too! its Unfamilia's game demo using DX10 FPSCX10

http://www.akaneaya.com/tgcforum/DOWNLOADS/TOF.zip
Approx~ 505MB

You can see the information at http://www.akaneaya.com/tgcforum/

Should be a good prolonged test for you all

Rudolpho there are ` marks in the example code above, should probably mention that users need to replace it with two // strokes [For the URLs]

I hope I can work this into one of my projects [The one I mentioned in the emails I sent you]

Hope this helps

EDIT

Also here is my code with my downloads pre-set



Tested on Windows 8 Pro x64 using DBP U77

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 15:03
Thank you, glad that it seems to be working
I'll edit out those "`"'s, annoying as I only replaced all instances of "//" with "`" since the code snippet sections won't recognize the double slashes as comments

If you could try it with a Vista installation that would be swell, I don't think there should be any difference on 64- and 32-bit machines as the plugin is 32-bit anyway (as is DBPro).
Also thanks for providing a decently sized file for download


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Dec 2012 15:46
I am using Windows XP and I gave it a try. I used a loop to queue up 392 files to download (my entire game so far with all the media). It downloaded the first 139 files, all the files after that stated "No such download exists". It does it every time. Is there a problem with that many downloads at once or is something I might be doing?
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 15:55 Edited at: 17th Dec 2012 15:58
Hm, I have never tried that many, granted.
get download status text and get download status description will however return "No such download exists" if the download in question has already finished, if that might be the case?
If you don't purge the cache for the files and / or if they are small in size they will be fetched more or less immediately which could result in it looking like that.


"Why do programmers get Halloween and Christmas mixed up?"
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 17th Dec 2012 16:04
I will install Vista tonight and give it a go at each Service Pack level... hopefully it will not jump straight to SP2...

hehe I already ran that test with the 1.5GB total a few times... good thing I have unlimited bandwidth ahahahhaha

Will post back as soon as

GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Dec 2012 16:44
Quote: " if the download in question has already finished, if that might be the case?"


I don't think so. They are small files, but I am using your flag to ignore the cache. The first 2 are large, and so all the rest wait for those two. The first two and the next 137 all download a couple at a time starting with ID 0 and moving up, but 140 and onward start off as "No such download exist".

I also copied the project to a Windows 7 PC which had never downloaded those files, and got the exact same result.
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 20:21
Hm... would you mind sharing your code GI?


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Dec 2012 21:21 Edited at: 17th Dec 2012 21:35


You would not believe how much trouble I had to make that example code. I kept getting "Illegal Operation" errors when trying to use the PRINT command...

Anyway, the above code asks how many files to download, and then downloads a TGC banner that many times. It saves the banner to a downloads folder with a number representing which download it was. It will list all the downloads, scroll using your mousewheel.

When I ran this, and entered 200, it downloaded 1-168, and the rest were invalid. Enter 75 or something lower, and it runs fine.

Edit: The above code requires Matrix plugins
Edit2: Just ran it again and entered 200. It said 176 were done and the rest "No such download exist", but it didn't actually download anything...

Edit3: At what point in time does your plugin delete all information about a download? Could we get a version where it doesn't, and we manually do it? I tried calling "get downloaded bytes" after a download finishes, but it says 0. My guess is that the download was already cleared out of memory and the bytes are long gone.
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 22:04
Thanks, I'll look at it

Most download information is gone as soon as the download finishes; each download runs in its own thread so as to make it possible to download several files simultaneously and it thus made sense to remove the downloader object when the thread finishes.
The actual downloader object isn't deleted until five seconds after the download ends however, but the reference from the ID map is removed immediately (this is to allow catching up in case a function was issued from DBPro before the download was completed but that didn't complete before it was).

It would of course be possible to make the download objects linger on and have the user manually delete them after they complete as well. Now that I think of it perhaps that would be a more logical approach.


"Why do programmers get Halloween and Christmas mixed up?"
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 17th Dec 2012 22:09
Unfortunately, I was unable to test Vista... the VM will not connect a network profile

Sorry, I could test in a few weeks on a physical machine...

GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Dec 2012 22:14
I have a vista machine behind me I could plug in and test on, but I will wait for Rudolpho to check over my last snippet and perhaps send out a new version.

I have been trying to find the best way to download via HTTP for a long time and have never really found a good way. I tried the URLDownloadToFileA method, the HttpQueryInfoA method, and recently was tempted to write something in DarkNet to communicate directly with the web server. Your timing on this DLL could not be better!

Quote: "The number of possible simultaneous downloads may vary from system to system."


What determines the number it can do?
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 22:19
Hm, this is just an early guess, but since you're downloading the same file simultaneously it's quite possible that the simultaneous requests to the url itself creates a situation that cannot be managed. Failing that, for each new download you're purging the cache of the same file, that might delete the just opened download stream. If that happens the previous download(s) will fail.


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 17th Dec 2012 22:40 Edited at: 17th Dec 2012 22:44
Here is new demo code that pulls images off my web server. Its the same image, but with 200 different filenames. If you enter any number above 200, the files above 200 will fail as they do not exist. However, entering 200 in the code below has the same result.


Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 17th Dec 2012 23:05
@MrValentine: I see, don't worry about it
Thanks for the effort though.

@GIDustin: This indeed seems to be an issue.
I've managed to get my (console) testing program to download each one of those 200 files you uploaded, so I guess I'll have to go on a difference hunt between that one and the plugin source...
I'll get back if I make any progress


"Why do programmers get Halloween and Christmas mixed up?"
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 18th Dec 2012 13:10
If you are downloading the same images... again and again... could be a memory cache issue... perhaps try 200 different images...

Try creating an app which creates 200 bmp images with different dimensions and colour values... upload them and then download them...

If you need FTP space for extended esting I am more than happy to contribute some...

I would do this right now but I am at my shop with next to no PC web access and usIng my phone as a hotspot is useless... I have the mains box in my shop lol...

GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 18th Dec 2012 15:11
MrValentine:

I used code similar to what I posted above to download 392 images, all different, that I use in an app I wrote for work purposes, and that is where it originally failed. The test code uses the same image because I am not sure I can really release those work images. Both my actual app and the test code above show very similar results. It is almost like the downloader DLL runs out of memory to create more download requests, and just marks them all as done or invalid.

If I get some extra time today, I will create differing images and replace the ones I already uploaded to my website.
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 18th Dec 2012 15:50
I see... yeah I agree it has to be memory allocation related...

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 20th Dec 2012 21:36 Edited at: 20th Dec 2012 21:36
Try this one out and see if it works any better.
Essentially it queues downloads to ensure that too many threads aren't run at once. If it works (it seems like it does, although I've had the odd session error code returned last night and am still uncertain what might have caused that; nothing such have happened today however) this concept could be further improved on.


"Why do programmers get Halloween and Christmas mixed up?"

Attachments

Login to view attachments
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 20th Dec 2012 23:17
That one doesn't seem to have the same problem as the last. I was able to get all the files to download.

However, if a URL was invalid, it would throw a popup instead of just returning the status text. Also, every download now starts as "No such download" and only changes once it's turn comes in the queue. I would think their status would be "N/A" or "In Queue" if that would be possible.

Did you change anything with how the downloads are cleared out of memory? For me, it seemed to run a little slower and I was wondering if that was because I wasn't running a "clear" command or something on completed downloads...

Great progress though, and thanks for looking at it!
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 21st Dec 2012 00:02
Yes, it's merely a test, hence the fugly error messages, nice to hear it seems to work though

Quote: "Did you change anything with how the downloads are cleared out of memory? For me, it seemed to run a little slower and I was wondering if that was because I wasn't running a "clear" command or something on completed downloads..."

Not yet, but I might do that next.
It runs slower because it only allows 16 download threads at once and each one sleeps for half a second before terminating to allow any lingering function calls to catch up before the downloader instance is free'd. This inefficiency problem would be solved by having the user delete the finished downloads himself (the worker threads would still terminate so no problems there, but the downloader object holding the last data set by the associated thread wouldn't be freed until a call to delete download or something like that is made).


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 21st Dec 2012 00:08
Quote: "It runs slower because it only allows 16 download threads at once and each one sleeps for half a second before terminating to allow any lingering function calls to catch up before the downloader instance is free'd. This inefficiency problem would be solved by having the user delete the finished downloads himself (the worker threads would still terminate so no problems there, but the downloader object holding the last data set by the associated thread wouldn't be freed until a call to delete download or something like that is made)."


Makes total sense. What about the downloads showing as invalid while they are sitting in queue? I think a new status should be added for those, such as "In Queue".

Can't wait for the finished product. If it works as well as the tests, this is going to save me alot of time on all my "Updater" projects out there. Might have to reimburse you somehow...
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 29th Dec 2012 21:15
Rudolpho:

Make any progress on this? I am anxious to get it plugged in and working!
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 30th Dec 2012 13:13
Sorry for my absense, I've been a bit busy with Christmas and some other personal projects.

Anyway, I began reworking the plugin this morning and hopefully it should be more or less done by now (still need to add some additional wrapper functions for checks, update the documentation etc.). I won't have any chance to test it properly until tonight however, but if I'm lucky I might have the new version out tonight or tomorrow.

Among the new features is user deletion of finished downloads, this should speed up the queue significantly (it won't have to sleep for lingering function calls as the DBP user basically says "this is my last call to this download" when requesting to delete it) as well as allowing downloads to return sensible data other than zeroes and N/A after finishing. There's also error code retrieving instead of the fugly message box.


"Why do programmers get Halloween and Christmas mixed up?"
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 30th Dec 2012 21:58
Alright, the updated version can be found in the opening post now.
It seems to work fine judging by my tests, no unexpected behaviour so far


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 30th Dec 2012 22:48
Just ran it on my project and it seemed to work fine. One thing I noticed is that about 15 were waiting in queue at a time, but only 2 were actually downloading. I noticed a set maximum download threads command which changes that 15, but still only 2 download at a time. What controls that other number? You mentioned earlier it might be due to the cores available, but is it something you can change?

One thing I noticed on my system is that it only downloaded 2 files at a time, so if I set the threads to a lesser number (4, instead of the default 15ish), the downloads seem to run much faster. If there was a way to find out how many downloads are possible at once, we could set the threads accordingly and increase the overall speed. So, if it isn't possible to set the number of concurrent downloads, is there at least a way to find out what that number is?

Anyway, from what I can tell it is working like a dream. I plan to implement it into my project Monday and send it out to 50+ PCs and see how it holds up.

Thanks!
MrValentine
AGK Backer
13
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 30th Dec 2012 23:45
Hopefully in a week I can run it through my toolset and get back to you

Thank you once more

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 30th Dec 2012 23:49 Edited at: 30th Dec 2012 23:54
Quote: "What controls that other number? You mentioned earlier it might be due to the cores available, but is it something you can change?"

I believe it is bound to the cores (as in on the hardware level, so no you can't change it with code). That means you would likely get one download on a single-core machine, 2 on a dual core and 4 on a quad core. I was unaware of this limitation at the time I wrote the thread distribution system obviously, but it definitively doesn't stem from my code so there isn't much to do about it besides rethinking the whole system I'm afraid. I did not think that having several threads all waiting would cause considerable slowdowns like you're describing however. That's odd

Quote: "So, if it isn't possible to set the number of concurrent downloads, is there at least a way to find out what that number is?"

Yes, it should be stored in the environment variable NUMBER_OF_PROCESSORS. You can retrieve it like so in DBP (requires IanM's plugin pack for the dynamic memory pointers):



Glad it seems to be holding up for you, hope it will continue to
Happy new year by the way


Edit: Thanks Mr Valentine, looking forward to it


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 31st Dec 2012 16:43 Edited at: 31st Dec 2012 16:50
Rudolpho:

Edit: Ignore that comment. I was using URLs wrong.

Just FYI, I was requesting files like this:

http://www.somedomain.com/Files\SomeFile.ext

And it seemed like the PurgeCache flag was broken. Once I swapped that last slash to a forward slash, the purge cache flag started working again.
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 7th Mar 2013 17:46 Edited at: 15th Mar 2013 17:25
@Rudolpho:

Trying to use your plugin for a new project, and it isn't working 100%. I looked over my code as much as possible, and cannot find the problem. Here is basically what happens.

Start downloading about 10 files. delete download as they finish. Once all 10 are done, wait a few seconds, and start another batch of 10. In my experience, those next 10 do not always finish, and when you query the first one of the new batch using GET DOWNLOAD STATUS TEXT, it says no such download exists, and all the rest of the new batch are "Queued".

If I get a chance, I will extract a few lines from my project and try to make a new test project for you.

Edit: Some code for you. I found some open webserver with a bunch of files, and set this up to download 10, then wait 2 seconds, then do the next 10. For me, it always gets stuck around 20. Any ideas?

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 9th Mar 2013 15:30 Edited at: 9th Mar 2013 15:31
Hm, that sounds odd... I would try it out for you but don't have access to DBPro at the moment.
A download will be queued if there are too many download threads running already, so this would indicate that your download processes don't terminate as they should.
You can try tampering with the maximum simultaneous thread count using the set maximum download threads function to see if that makes any difference.


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 9th Mar 2013 18:21
It seems the queue works fine as long as it never empties. I can load the queue with 900 files, and it downloads them all just fine. But, if the queue becomes empty for any decent amount of time, it is like it falls asleep and won't wake up.
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 10th Mar 2013 00:13
Hm, looking at the plugin source I have here (which I believe is the latest version at least) a single download process is popped from the queue whenever a previous download finishes.
This might be able to cause issues and a better solution would be to start as many as possible (up until the max count is active again or none remain in queue) after each finished download. It most likely shouldn't cause a complete freeze however.

What you're saying about the problems arising when the queue runs empty doesn't seem to make much sense at all to me (judging by the plugin code) but I'll try to look into it.


"Why do programmers get Halloween and Christmas mixed up?"
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 10th Mar 2013 04:47
Try the last code snippet I posted. For me, it stops after 2 iterations every time.
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 14th Mar 2013 19:53 Edited at: 15th Mar 2013 17:41
@Rudolpho:

Trying to track down this problem even further to help you out. I have another script that queues up 100 files. It always downloads the first 16 files and then stops. When I run a quick script that checks the status of each file, I find weird results:



Outputs:


You will see that downloads 1-7 were properly deleted, 8-15 are "aborted", and the rest are in the queue. I can guarantee you that abort download is not in my script at all, so I am not sure how those are getting aborted as opposed to deleted. They are what is jamming up my download queues.

I then tried running this every couple of seconds:



And it seemed to not matter. They still stayed active, and still marked as "Aborted".

Hopefully that helps...

Edit: Updated my previous post containing an entire test program to show the same thing, several files are "Aborted" and holding things up. Basically, it seems that when you "delete" some downloads, it marks them as "Aborted" instead, and they still take up an "active download" slot, so no new ones can start.
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 20th Mar 2013 13:02
Thanks for your help, I think I managed to find the problem because of it
Try the attached version and see if it behaves better.


"Why do programmers get Halloween and Christmas mixed up?"

Attachments

Login to view attachments
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 20th Mar 2013 14:08
My test project ran all the way through without any problems. I will try the new DLL in my main project later today, but it looks as if the problem is solved!

Thanks!
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 20th Mar 2013 15:17 Edited at: 20th Mar 2013 20:06
@Rupolpho,

My main project still has the same problem. I added a bunch of debug code to see what is happening. I do is download finished which returns 1. I then check get download error which returns 0. I then call get download status text and it returns "Finished". So I delete download. Immediately afterwards, I call get download status text again, hoping it will say "No such download exists", but instead it says "Aborted". This happens on about 15% of the downloads, and causes the download queue to stop operating until I start another download, which then kicks it back into gear until another download is marked "Aborted".

Any clue what might be causing that?

Also, while the download is in progress, is it writing directly to the local file, or is it writing to a temp file, and then moved when finished?


Edit: The only difference between my main project and the test project was that my main project set maximum download threads to 8, whereas the test project left it default. If I add a set maximum download threads to my test project, the "Aborted" downloads show up there too. I took that command out of my main project, and the "Aborted" error comes up much less often, but it still does...
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 21st Mar 2013 15:07 Edited at: 21st Mar 2013 15:09
Hm... that sounds weird.
You can try using the function get download process count() to see if the "aborted" ones still are running.

Quote: "... Any clue what might be causing that?"

If a download that is in the waiting queue is deleted it will be marked as aborted. It seems like the "isInQueue" flag was never reset once the download gets popped. I've fixed that now; try the attached file

Quote: "Also, while the download is in progress, is it writing directly to the local file, or is it writing to a temp file, and then moved when finished?"

It writes to your temp folder.



Edit: Obviously I started writing this post before finding and addressing the flag issue if someone thought it looks weird :p


"Why do programmers get Halloween and Christmas mixed up?"

Attachments

Login to view attachments
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 21st Mar 2013 16:26 Edited at: 21st Mar 2013 18:12
@Rudolpho:

I was about to complain and say the newest version won't download a single file, but it appears the random website I was downloading from is having internet issues.

I think we are almost there, but not quite. I have a test project that I tell to download 40 files. I added your get download process count(). It starts out with 16 processes, and stays at 16 until the number of downloads remaining gets to 16, then each decrease at the same time until they both reach 0, and we are done.

I then put that in my main project, and I see different results. When I queue up 40 files, it starts with 16 processes. About every 2-4 successful downloads, the number of processes decreases by one. When there are 11 downloads remaining, the number of processes reaches 0, and the entire queue stops. If I queue up 3 more files, it sets the processes to 3, and downloads those three file, and 3 of the ones that halted earlier, before the processes reaches 0 again. If I keep queuing up files, eventually it will catch up and all the files will download, but I have to keep downloading files to start the queue back up again.

I think it is like the files that used to just abort (Which you did fix, they say they don't exist now) are still removing a process even if there are more downloads waiting. I can't say for sure since I don't have your code, but that is my best guess.

Edit: I cannot get any of my test programs to do the same thing. Going to put a whole bunch more debug code in my main program to see if I can narrow it down...

Edit2: I tried one thing different and it seems to have fixed the problem, but caused another. After every download file I put a nice wait 100 and it seems to have fixed the problem with the download processes. My guess, and again, I don't have your code so I could be shouting out groundless theories here, is that it has something to do with the processes. I know that DBPro is not capable of having different threads because of the possibility of two threads accessing the same memory at the same time. Perhaps something of that nature is happening here. Either way, it seems that some times the process count decreases faster than the number of downloads. But, having that wait command really slows down the program.
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 25th Mar 2013 21:38
Sorry for the double post.

@Rudolpho: The problem seems to only exist if you download many files with the [autostart] flag set to 1. I changed my code to download with autostart=0, and then begin download on each of them directly afterwards, and it seems to work now.

Login to post a reply

Server time is: 2024-04-19 01:24:28
Your offset time is: 2024-04-19 01:24:28