Why do browsers still not have file upload progress meters?

It’s 2008.

Firefox 3 was just released – years of work, thousands of bugs fixed, new features thrown in.  No file upload progress meter.

IE7′s been out for awhile, but I don’t have a copy here handy to test.  My memory is telling me it doesn’t have one, or if it does, its very unobtrusive and not immediately apparent.

Opera 9.5 was recently released.  No file upload progress meter.

Konqueror 3.5 – no file upload progress meter.

It’s too late for me to check Safari2 – but I don’t think it’s got one.  I don’t have Safari3 on a Mac, so *maybe* it’s got one, but I doubt it.  I think I’d have heard about it if it does.

This current tirade stems from implementing a file upload progress meter in PHP5.  Yes, PHP5.2 has some hook, and there’s a PECL extension.  However, there’s *0* documentation on it, and the few pages Google turns up are mostly people saying “it doesn’t work!” and a few “hey it works!” with little in between.  I have one other option, I think – the APC cache system apparently also has file upload progress functionality in it now.  I was hoping to avoid it, because 1) I think it might have conflicted with the ioncube encoder on the machine and 2) I was really hoping to use an extension that just did the one thing I needed, not a bunch of other stuff as well.

I realize this is partially a PHP issue I’m ranting about, but it’s ultimately a hacky workaround to a basic piece of functionality that browsers should support.  The browser is pushing the file up – it has a notion of how many bytes have been pushed out to the network.  It is in a position to know the accurate size of the file in question.  Showing a local status of that information would be a faster experience, rather than adding a layer of AJAX calls to an upload screen.

I was at a developer panel in Ann Arbor back in 2003 and someone from MS was on the panel.  He had been on the IE team at one point, and I remember making the suggestion to him to pass the ‘upload progress bar’ back to the IE team.  It was a bit of an odd response – it wasn’t a ‘no’, but a bit of a bemused look, as if to wonder why people would want that.  He didn’t *say* that, and was cordial enough to take the suggestion, but it obviously did bugger all.  I was sort of figuring IE would have included it in IE7, then FF would eventually make it available as an extension, adding yet “one more extension” to the list of ‘gotta haves’, then they’d talk about how innovate FF is.  :)   Or perhaps even Opera might just slip it in under the radar in the past 5 years.

Alas, it seems that people that write browsers are somehow oblivious to the need app developers face in this area.  Perhaps they all live on petabit connections and/or only do development with local servers, and uploading 10 meg files takes 2 seconds.  In the real world, for most of us, it’s considerably longer and more painful, both on the development side testing, and for end users uploading.


I'm currently working on a book for web freelancers, covering everything you need to know to get started or just get better. Want to stay updated? Sign up for my mailing list to get updates when the book is ready to be released!

Web Developer Freelancing Handbook

Share and Enjoy:
  • del.icio.us
  • DZone
  • Facebook
  • Reddit
  • StumbleUpon
  • Digg
  • Simpy
  • Technorati

Tags: , , , , , , , ,

{ 70 comments to read ... please submit one more! }

  1. Have you seen the Yahoo User Interface’s uploader component? Yes, I know it is flash but it enables web developers to create a progress meter pretty easily. And it is supported on all modern browsers.

  2. Tore B. Krudtaa

    Finally someone that finds this interesting.

    Why not post your interest in it here, and eventually propose a better feature request than I have done. (Maybe you know which buttons to push to make something happen ….)
    http://bugs.php.net/bug.php?id=39447&edit=2

    I have no idea why some core PHP developer have not yet used the hooks to provide a solution for people not using APC. It is quite a while since Rasmus made those hooks available.
    How long shall it take before we can use session variables to handle this?

    I would like to see the upload hooks available for usage within a normal PHP installation without the use of any quirky “add-ons”.

  3. do a search for “fancy upload” its a script written by digitarald. Supprts both multiple file upload as well as progress meter. Uses flash as well. I just implimented it and love it. Also needs mootools. Sure its a workaround but does the job quite well. Sorry for misspellings, sent from my iPhone.

  4. Tore B. Krudtaa

    Oops… forgot a couple of things….

    Personally I would rather have a solution in PHP, because then it would work crossbrowser.
    And despite the currrent lack of interest from php developers in making the upload hooks more available, I belive it is easier to get a solution for this within PHP since it is only one devteam that has to do something, not a bunch of browser dev-teams.

    Another reason for wanting to have a solution within PHP is that then I could probably, while the upload is in progress read the status of the upload and do whatever I want during the upload…. such as abort the upload if needed…. In short… I would get more control of the upload process and I could also then have more control in the feedback to the client.

  5. Galeon the gnome web browser has an option that show how much the data is loaded at the status bar. But it is not a progress bar :)

    http://ul.gcg.gen.tr/x/1b9b378.png

  6. Some websites, like YouTube, have an upload progress bar. It never seems to work for me as well as I’d like it to. I would totally welcome a browser that has it. I’m surprised nobody’s even made an add-on, extension, or plugin to do this.

  7. I agree with you that this should just be a feature of the input type=file.

    For now though, you can get your MooTools skills out of the closet and use FancyUpload, a great solution that uses a flash applet to handle all the uploading and progress combination hooked to javascript events. It also does multiple file @ once opening, etc.

    I’ve implemented this successfully at 3 sites now and its just awesome.

    http://digitarald.de/project/fancyupload/
    Don’t wait up for the browser-dudes! this works *now* :D

  8. I can confirm APC and the ionCube loader will both exist in harmony on the same server. This is how my servers are configured (although one now uses XCache instead) and I wouldn’t like to run without an optcode cache any more!

    A browser-based upload bar would be a nice feature but… would they be able to make it any more accurate than the download bars, where the time for the download to finish doesn’t match the actual time it takes?

  9. I completely agree with you on the necessity of a file-upload meter. I think however for it to become a browser feature it needs to become part of the standards first.
    The upcoming HTML5 specification has new API’s to many techniques that are not supported by current browser directly, such as a notification API (ie – COMET).

    Maybe you should propose a file upload progress API for the HTML5 draft

  10. You strike at Firefox for publishing these kinds of things as extensions, critizicing it for depending on those to be innovative. Yet Firefox is not a browser only for developers, -who by the way use mostly ftp to upload seriously- so the file upload meter is not an urgent need for many.

    I really don’t see the ‘need’. If you are a developer, you can do a back-o’-envelope calculation with file size and connection speed, hopefully known quantities. Or assume that uploading big files will take a while (the standard while is at the institute of weights and measures in Paris). Having an upload meter won’t reduce the time needed to upload a big file, after all, it will only soothe you while doing it.

  11. Actually Opera most of the times (not in Gmail) shows progress of file uploading (or at least it was doing so last time I checked)

  12. I’m not sure if you’re referring to the PECL extension I’m thinking of or not, but I know that Ben Ramsey worked on one for the same purpose. You might try pinging him on IRC or via his web site if you need help with it.

  13. There’s even an open ticket in Bugzilla to get it into Firefox but alas, no change. I agree, though. We have download progress, why can’t we have upload progress?

  14. He mentions that the browser knows how much of the file has been sent. Sure, but how do you know it’s been received on the other end? Edit: Only the server knows that (right?), and for the browser to get that information and turn it into a progress bar, you’d need to redesign the whole uploading mechanism.

  15. Rasmus Abrahamsen

    I agree with you that it would be best to just implement it in the browser. But have a look at lighHTTPD, the version soon to come out will have this feature supported. It also supports PHP with APC ofcourse, but the web server will have it built-in.

  16. @sunnybeach
    If the server’s not receiving things, you likely have bigger problems than uploading – like, the browser wouldn’t have worked because the server wouldn’t have received the request for the page with the upload form, right?

    @Tomek
    I’d just tried a file upload (not in gmail) on opera 9.5 (linux) and it didn’t show anything. Perhaps on other platforms it does?

    @Adriano
    You don’t see the need for it? Any project I work on where site users upload files (videos, pictures for avatars, etc.) it’s a requested feature from clients. Telling clients to tell their users (via FAQ?) how to calculate how long a file *should* take to upload is just insane.

    @Eran
    It’d be OK if it was a “standard”, but don’t particularly. We all see how well those have worked out, right? There’s *never* any room for ambiguity or misinterpretation in “standards”, right? :) If it was to be programmable via javascript (I’d like that), I’d tend to agree, but I don’t think it even needs to go that far right now. Download progress meters, browser tool status bars, and other browser visuals are not defined as part of a ‘standard’, yet are implemented in browsers. This is something which can purely be informational, and wouldn’t really require any committee agreement. It might be *nice* if it did, but I’m sure there’d end up being ‘published specs’ that no one followed to the letter and we’d be back to another incompatible feature anyway.

    @many
    If I have to go for a Flash version, I will, but was hoping to avoid that. Thanks for the suggstions.

    @tore
    I’d rather have a few major browser teams implement it (IE, FF, Safari, Opera, Konq) over all the major languages having to implement it. Perl has a way to accommodate it, but there’s no ‘built-in’ way in .net, java, coldfusion, php, ruby, etc. Looking at it, it sems Ruby on Rails has a mechanism for showing this. Old classic ASP needed external components, I remember that.

    FWIW, the problem I seem to be having is that uploaded files are still being written as /tmp/phpHerjFHR file names, when it seems they need to be written as /tmp/upt_UPLOAD_IDENTIFIER (where UPLOAD_IDENTIFIER is passed in in the form). The file upload portion of PHP doesn’t seem to be doing this. At least, that’s my understanding of the PECL version’s issue.

    If there’s something as simple as having to php something like upload_rfc1985=on or similar I’ll scream this morning, because, as far as I can see, that’s not documented ANYWHERE. I do see that the APC version needs to have that in php.ini. (Thanks to whomever mentioned that APC and Ioncube can coexist – I may give that a try this morning after breakfast).

    Thanks all for the feedback and discussion so far.

  17. The PECL extension (uploadprogress) works, as described by some helpful commenter in the notes section. It does trash the data as soon as the upload is complete; I had the code receiving the POST set a session variable to indicate to the AJAX-powered GET side that the upload was completed, so I could distinguish “no data because PHP didn’t feel like updating it yet” from “no data because the upload is done”.

    Even so, a hook exposed in JS so that you could just ask the browser how much was uploaded would be far preferable to the current situation. I don’t think browsers are planning on supporting a common upload progress UI themselves, and even if they did, someone would want to skin it anyway.

  18. Clever point, Michael. I hadn’t thought of this, but of course you’re right. It’s about time.

  19. Opera shows UL/DL progress in (surprise!) progress bar.

  20. @sunnybeach,

    The tcp stack will throttle the uploads based on acks from the server. We can assume that bytes sent is pretty close to bytes received by the server, give or take a few packets.

  21. This does seem like a client side issue. I mean the client knows exactly how big the file is and how much of it has been sent. Therefore the client has everything it needs to make a progress bar. It it just lacking in html/javascipt/browser, perhaps there needs to be an html element to output it or be available from the DOM. Call ISO… :O

  22. Using php (or any other server side language) for this task is not best choice – files are sometimes cached before sending it to script so you have no chance to check progress.

    There are upload progress modules for Nginx, Lighttpd and Apache.

    http://wiki.codemongers.com/NginxHttpUploadProgressModule
    http://github.com/drogus/apache-upload-progress-module/tree/master
    http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back

    I wrote an article about apache upload progress module. http://drogomir.com/blog/2008/6/18/upload-progress-bar-with-mod_passenger-and-apache

    All you need is a bit of javascript. It can be used with any language/framework.

    It’s sad that we still have to code such things and there is no support from browsers (for example upload progress API), but with jQuery and those server modules implementation is pretty simple.

  23. file transfer progress meters are actually based on the amount of data received by one party. when you download, it’s your computer that counts the data.

    when you upload, it’s the server that counts the data. data packets might be lost on their way to the server, so the actual amount of data that has been uploaded by your browser is completely irrelevant.

    so it’s not really a browser problem. it’s a server problem. apache can be extended to support this, but it’s not in the default configuration. but apache is only one of the dozens of web servers avalaible out there. Web developers also have to tweak in with ajax or swf files to allow this. Don’t also forget to factor in people who develop in .net, ruby, python etc.

  24. Last time I checked, the APC upload progress meter was not thread safe…

  25. Well unfortunatelly i have never seen a Upload function “from user view” which i really liked most of them doesn´t work that good even on large sites where upload is main business like youtube, googlevideo or flicker. Sometimes the ajax stuff crashs or just doesn´t work without reason.

    Even it is not only a PHP problem it is a web problem specially if it comes to upload multiple files.

    You always need a trick like extra programm, .exe or java solution.

  26. The thing I wish is that firefox would not freeze and stop all other activity while you SAVE a page to your hard disk.

    anyone know a work around for that? I can’t believe all resources must be stop to support a simple “save page” command.

  27. People saying that this is “a server problem” because the uploads are completed at the server site are totally missing the point.

    1) It is extremely difficult to implement in the server
    2) The users don’t really care about the difference between how many bytes are sent and how many bytes are received
    3) Making the difference between send and received is extremely pedantic. For instance, users consider their mails “delivered” when the client says they are “sent”, even if the time difference is sometimes several minutes. It does not cause problems.
    4) The difference between bytes sent and bytes received is very small, and negligible in case of big downloads (which is the interesting case for a progress meter)
    5) The browser can easily pretend that the last bit of data isn’t sent until the POST succeed, so, in practice the difference would be undetectable

    That is not rocket science, and is just another case of developer hubris.

  28. Given that 99% of the time I use scp to upload files, and .99% of the time I use WebDAV from a folder on the desktop (OS X does some strange things), I can’t really say that the file upload meter issue is a significant one for me.

    I agree, however, that it belongs as a feature of the browser. Not a JS hook, a flash hack, or a server-side feature. The browser has the information, and the browser has the user that is interested in the information, therefore the browser should do the work, rather than having the browser provide the information to the server and relying on the server to tell the browser to tell the user what the progress is.

    I’ll have to set up a file-upload target on a page somewhere, and then upload some sizable junk files to test various browsers, just to check ‘em out for myself, even though it’s not an issue that’s anywhere in my top 10 problems with modern browsers.

  29. “Yes, I know it is flash but it enables web developers to create a progress meter pretty easily. And it is supported on all modern browsers.”

    Uh, yeah. Sadly, Firefox 3 on my Core 2 Duo apparently isn’t modern enough (unless I want to install a complete 32-bit system in parallel, but, uh, no thanks). But if I upgraded to a nice ol’ Pentium II 800, that would work. Or if I went on ebay and got a G3 Mac with Safari 1.0, that would be “modern” enough, too.

    When you claim that something is “supported on all modern browsers”, when in fact it’s only supported on the set of platforms which Adobe Flash Player supports, you’re screwing over some of your users.

    Whether you know and/or care, that’s a different question.

  30. maybe I’m just stupid but when the server receives data doesn’t it send an ACK? If the people who are saying “bu-bu-but the browser doesn’t know when the file is received on the server” are right, then how can uploads EVER complete successfully?

  31. I think this should be implemented by the browser vendors… all the these hacks create a unconsistent user experience for something so tied to the users browser.

  32. Why do browsers still not have file upload progress meters?

    Which reminds me of the fact that browsers don’t have some kind of “volume control” either, which in fact should be there to have control over your multiple windows multimedia experience…

  33. @heri
    when you upload, it’s the server that counts the data. data packets might be lost on their way to the server, so the actual amount of data that has been uploaded by your browser is completely irrelevant.
    so it’s not really a browser problem. it’s a server problem. apache can be extended to support this, but it’s not in the default configuration.

    This is totally stupid. Your code does not have to resend TCP packets if they get lost in transit. That’s handled by the network stack on both ends. Trust me, just a progress meter that shows the number of bytes returned by fwrite() would take care of the entire issue. fwrite does not count network overhead..

    Any solution is better than no solution, or haphazard server side implementations. Nobody expects progress bars to be exact, in fact, I can’t remember the last time a progress bar didn’t hang at 99% or disappear right when hitting 100%. So, a requirement to be exact is just a red herring.

    For all this talk about how easy it is to do with jquery + apache/php modules… I’ve still never seen it done. Where are the examples of this so touted “easy” technique?

  34. Uh, yeah. Sadly, Firefox 3 on my Core 2 Duo apparently isn’t modern enough (unless I want to install a complete 32-bit system in parallel, but, uh, no thanks). But if I upgraded to a nice ol’ Pentium II 800, that would work. Or if I went on ebay and got a G3 Mac with Safari 1.0, that would be “modern” enough, too.

    When you claim that something is “supported on all modern browsers”, when in fact it’s only supported on the set of platforms which Adobe Flash Player supports, you’re screwing over some of your users.

    Whether you know and/or care, that’s a different question.

    There’s only 3% of you out there who don’t have flash player…. get with the times!

  35. Also, if it’s impossible, why does IE do it? Nobody notices because it’s just a tiny bar in the bottom right of the browser, but they do their best at an attempt. Personally, I think something bigger would be more useful. How come the download utility is it’s own window with it’s own set of controls, but upload is relegated to a tiny progress thumper bar…

    Konqueror has one foot in the water. When you submit a form that has an upload it asks you in a confirmation dialog with the absolute path to the file just so you can know that some sort of file upload is going to happen, and that any long waiting period is not just a server timeout.

  36. @heri Normally I’m not so blunt, but there is no room for interpretation here, you are flat out wrong. There is no way for some packets to be missing when you upload a file without the whole file transfer failing. This is the entire point of the TCP (Transmission Control Protocol) in TCP/IP. Without TCP the Internet as you know it wouldn’t be possible. You could play Quake, but you couldn’t download a file. You could listen to streaming radio, but you couldn’t send an email. If you are suffering unrecoverable packet loss then the connection is nowhere near good enough for the server to send any information back to the client about upload progress anyway, thus nullifying any possible benefit of a server-side solution.

    I’ve actually implemented one of the available AJAX polling-based upload progress bars. To make a long story short, it was unnecessarily complicated, had bad scalability characteristics (ajax polling?), and the resulting bar was not particularly smooth in the naive implementation.

    Our team ended up implementing SWFUpload instead. Yes, using Flash is far from ideal, but it is by far the easiest implementation resulting in the best user experience. I’d rather give this experience to the 95% of people that can experience it and just have a dump HTML upload script with no javascript and no flash for the remaining 5%.

    All that said, it is sort of shocking that browsers still don’t have any kind of upload progress indicator after all these years.

  37. Wow, I’m really surprised to see this discussion here. I implemented the upload progress functionality in PHP, about a year ago. I had no idea other people were still having problems with it. The only configuration I had to do was install the uploadprogress extension and enable it in php.ini. (extension=uploadprogress.so)

    I think one of the trip-ups is that some people think that you can check the progress from the script that is handling the form/file upload. That is not the case. That script only starts execution once the file has finished uploading, just like it always has in version prior to PHP5.2. You have to check the status of the upload in PARALLEL to the actual file upload.

    My particular implementation doesn’t use AJAX but does use Javascript; it sets the form target to one hidden iframe, and on form submittal, it loads the ‘progress monitor’ script in another iframe. The ‘progress monitor’ script uses the uploadprogress_get_info() function every 0.5s and spits out a javascript command (followed by a flush() ) which updates the progress bar on the parent/main page. Works great. It’s actually possible to get the update interval down to 0.2s without any additional load on the server.

    But in any case, good article, bringing up an interesting point.

  38. If you ever bothered to look, most websites have the progress bar done for you.

    Then look further below. See that “progress bar” at the bottom right corner? That does a fairly good job of telling you when everything you’re doing is completed.

    So, in reality, it’s been there since Windows 3.1, you’re just not looking hard enough.

  39. @Alex – Speaking for myself, and I’m unanimous in this, what on earth are you talking about?

  40. I totally agree, this is something that should be done in all browsers. The browser knows how big the file is, and how much it has sent. It could be almost identical to the current download panels, or a modal upload box.

    The other ability browsers should have, which would not be as simple to implement, is to upload multiple files at once eg. a directory of images for a gallery.

    It seems so obvious.

    Using software based uploaders is not a good longterm solution, it makes all sites inconsistent, and they require the use of flash for the good ones.

  41. He means that the normal “page progress” bar works for both upload and download. When the bar reaches 100% your file has been sent.

  42. @olli – it’s not there on browsers I use. I’ve seen that little ‘page progress’ thing move around in years past on some versions of some browsers, but it gets overridden by other events, is not standard or consistent, is not readily apparent at all as to what it’s doing, and – oh yeah – did I mention inconsistent?

    If it was “there” and I was just “not looking hard enough” as Alex states, why do 20+ other people on this thread seem to agree that it’s not there? If 30 people had all written and said “you’re stupid, it’s there, wake up, etc” I might think differently. Considering I’ve been doing web app development since 1995, I think I know enough about browsers and how they work to know that this isn’t a useful, standard feature, nor is it “there” in most browsers.

    Please everyone, email me screenshots of browsers with the feature in action (mgkimsal@gmail.com).

    Thanks.

  43. on this screenshot anyone can see opera’s indication of upload progress. I’m uploading 30MB of data and 14MB are already sent.

  44. According to comments in bug 249338, it used to work in Firefox, and stopped working some time in 2004.

    Firefox’s networking code has been under-owned for some time since Darin Fisher was absorbed into Google. Hopefully someone will step up and start fixing networking bugs soon.

  45. @arty – thanks. Very interesting, because it’s not in my Opera 9.5, just downloaded a few days ago. Is there a setting you had to change to get that displayed? Like it said before, this is, at best, inconsistent among versions, let alone browsers. I’d also suggest that the location and prominenece of that is small enough such that many people might miss it. The counter to that is that I’d think most people using Opera are a bit sharper, simply because they’re early adopters of new tech and likely can spot that whereas an average person can’t.

    @Jesse – thanks for finding that. I knew it was in FF at some point, but can’t remember when I stopped seeing it. It’s so odd – I don’t even see any input from devs as to why it changed or why this issue doesn’t get any attention. If there’s a technical objection, let someone at least voice it!

  46. This toolbar is called “Progress bar” and can be displayed in 3 ways: inside address bar, popup at bottom, and “Simple”. I believe “Simple” is on by default to reduce information overflow for inadvanced users, and in this case it displays lesser amount of data.

  47. I tend to agree with my esteemed former colleague. It is surprising that a standard, built-in mechanism has not emerged from the technological jungle (or is it a swamp?) that is the Internet. A related question might be, why is there apparently no standard server-side implementation of RFC 1867?

    I guess the reason this feature has not become de-rigeur in all modern web browsers is because it requires some sort of standard “handshake” between client and server. This has led to the evolution (?) of products like SWFUpload, aurigma ‘s ImageUploader and apache’s FileUpload java package. At this point, most users have become so accustomed to these sorts of upload solutions that building one into a browser might actually confuse them. And often, one needs an upload interface tailored to the needs of one’s application. Even so, it would be nice if we didn’t have to keep re-inventing this particular wheel. There are much bigger and more interesting fish to fry.

  48. “Esteemed”? :) Self-esteemed, maybe!

    There’s rfc1867 ‘hooks’ in php5.2 and up now, and may be in other platforms. I still maintain, though, that this is something ultimately best handled client-side only. If you want to AJAX it to talk to the server and get some sort of progress how the server’s doing, go ahead and use that route.

    We’ve seen a load of work in the FF3 download manager system, yet there’s absolutely no way for a server (at a standard app level like php or java) to know if the file’s really being downloaded. We don’t poll the client and ask “how many bytes did you receive?”. The server sends the file, and an assumption is made that the other side is receiving correctly. And likely 99+% of the time, it’s a correct assumption.

  49. As a followup, I’ve tried to look at ‘fancy upload’ at least one person above suggested, and I have to say it’s exceedingly undocumented. The main docs seem to be a link to ‘forums.mootools.net’ which doesn’t exist anymore, and this gem from the main project page:

    How to use

    Will be written for the final release, at this time the showcases are the best documentation.

    Above that are about 15 lines of text about how awesome the tool is, and later on we see all the bug fixes, but no example code.

  50. I use to upload pictures to a host called iimmgg.com that has some kind of feature to enable this browsers upload progress bars, so it seems like browsers DO have support for upload progress, the problem is that it is not well documented and that’s why nobody is making use of it.

    Also some video uploading sites make use of this feature.
    Cheers

{ 3 Pingbacks/Trackbacks }

  1. Блог без заголовка » Silicon Valley Report in Russian » Вещи, которые броузеры не умеют дела
  1. Lessons learned from a reddit overload | Michael Kimsal’s weblog
  1. Links for this week : Peter Breuls’s Weblog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">



0.34964108467102