Assyncrhounous data loads on flex charts

[kminev]

Hi,

I have a flex chart which load very very large amount of data (50 000 - 200 000) records per load. Is there a way I can set my web service on the flex level to load data assoynchronously or can I do some kind of caching to make the users wait time either minimized or perhaps let the user see the data while loading some more data on the background.

Any tips will be appreciated.

Thank you.

[fathwad]

Afaik, flash player is single-threaded with a few exceptions (I think uploading/downloading is one of those exceptions, if that helps...I'm not sure if this applies to loading from a remote call). What I have done before was ask for user interaction AFTER an upload has been initiated. It worked well since it made the upload time seem a lot shorter.

For your situation, you can try simply breaking up your calls. i.e., load 1000 at a time and do processing in between to show the user progress.

[kminev]

Your idea about showing a progress bar on every 1000 records or so is a good idea, but it would be ultra cool if there is a way of caching data on the flex level. I know there is something built in to it, but not sure how to utilize it. thanks

[Sleeve]

kminev-

Look into the SharedObject for storing data locally on the users computer. I think you'll find that loading that many records from a remote call will lead to the app 'not responding' until the flash player parses all that data. Also, if the user is on a slow machine, the player may throw a timeout message and ask the user if they want to stop running the script which would be bad.

[fathwad]

I don't mean to undermine your advice Sleeve, but I don't think Shared Objects would work well in this context. SOs are roughly analogous to browser cookies so using that to "cache" would be a bad idea - just as saving the results of a remote call into a cookie would be. I believe the timeout is 15 seconds or so by default, but yes, there is a timeout. Using setTimeout as I mentioned won't cause a timeout on the client side since the process releases control in between each iteration.

e.g.

load1000() {
   //something
   setTimeout(load1000,5);
}

where 5 is some arbitrary time (that could be 0).

The timer is reset each time that function is called.


As for something built into the player, I can't say I know for certain. If you are just downloading something, you might find something in the Loader class. As I mentioned earlier, I believe this is a separate thread that won't timeout since it is not processing. This also lets you do other things (as expected from another thread).

From docs

var ldr:Loader = new Loader();
var url:String = "http://www.unknown.example.com/content.swf";
var urlReq:URLRequest = new URLRequest(url);
ldr.load(urlReq);

You should then look at the loader info (ldr.contentLoaderInfo) to generate your progress bar. Also, user interaction is possible I believe. To retrieve the data, listen to the complete event and access ldr.content.

[fathwad]

Forgot to mention, one disadvantage with using that Loader solution is that I don't think you can access the data while its loading.

[Sleeve]

byte arrays, zlib compression and shared objects - you are all set up to 400k+ with compression.

..and shared objects are VERY different from cookies. Don't you read the documentation?

[drkstr]

I'm with Sleeve on this one. Using a SharedObject with a custom loading framework is the best way to go. You can store quite a bit in a SharedObject. In my opinion, it would be an elegant solution to load and display the data in chunks. Load the first 1000 sets and hold it in memory and diaplay it to the user as soon as it's ready. Then behind the scenes, keep loading data in chunks of 1000 (or whatever you like) and store the chunks in a SharedObject. When a user changes the view, swap out the data held in memory with the data stored in SharedObject. If the data they want to view has not been loaded yet, they should be presented with a progress bar and that chunk should be queued to load next. As before, display the data when ready.

In my opinion, this would be an ideal application design. I'm sure you can get some help on this from many of the posters here if you need it.


Best Regards,
~Aaron

[creynders]

There really is no reason whatsoever to complicate things and start storing stuff in SO's. Just keep the data in memory. (And yes, do load them in chunks.)
Trust me, the swapfile communication runs a lot faster and smoother than SO-access.

And IMO SharedObjects are very much alike to cookies. It even says so in the documentation:

Local shared objects are similar to browser cookies

granted remote sharedobjects are quite different, but that's not what we're talking about here.

[drkstr]

Of course it would be easier to just keep all 500K+ records in memory, but that wasn't what he was asking. And yes, SharedObject is SIMILAR to cookies in that they both hold persistent data on the local machine, but the similarities end there. I don't remember off the top of my head the exact amount of storage, but it's in the hundreds of megabytes of binary data. That should be more then enough for a caching framework if this is the route you decide to go with. Otherwise, you can just let your app eat up all the available memory until it has to hit the swap file, as creynders suggests.

Best Regards,
~Aaron