I have a project that I'm working on where a user uploads a big file which gets parsed by the backend, and then returns the data back in a friendlier format. Now I'm wondering whether I actually need a database to store it, or should I just send all the data back to the user in JSON and let JavaScript to show the data properly on button presses?I'd like to add a database and have all project organised MVC, but I feel like storing the data and then extracting it to serve it to the user and querying it at each button press is unnecessary and would slow down the communication.I am aware that using the no-database method would mean other users can't access the data that the user who uploads the file can see, but that's not a big issue. Any other issues that I should consider?
Should I omit database and server communication (after initial response)?
web development;database;web services;web;json
It mostly depends on the size of the data you return and whether the user is expected to use all the data at once.For instance, if it's a list containing hundreds of thousands of complex entries:The response served as a single JSON will be rather large, and:It is unlikely that the user will actually need to see all the data at once.Instead, the user will prefer accessing the data either with paginated table or a table limited by criteria or based on a search, or in a form of a report which can eventually be configured on the fly. In those cases, there would be not one huge JSON response, but multiple JSON requests, serving a tiny part of the overall data which was collected or generated.On the other hand, it may be that there is no much to return and that the user will expect to see everything at once. In this case, multiple JSON requests are not really useful, and may even cause perceived slowness.If this is an important project, make sure you work with qualified interaction designers who can advise you according to the actual situation.ExampleTake an example of an application which let photographers submit a JPEG image and shows them the EXIF data. You determined, together with your users, that there are two sets of EXIF information: one set contains the data the users absolutely need: such as focal length or ISOthe ordinary photographers' stuff they manipulate on daily basis; another set contains the data that might eventually be useful: interoperability index, bits per pixeltechnical stuff many photographers will ignore during their whole career.Let's assume you did a prototype which returns all EXIF data at once, and it appears that it is too slow. Indeed, some photographs contain a lot of data which may be problematic to return in a single JSON response.So you decide to return the first set immediately and put a button See more which loads the remaining EXIF information. The second prototype appears a failure as well: users who actually need the remaining information find your application very unresponsive: they click on See more expecting the data to pop out immediately, and instead find themselves watching an animated loading GIF for a second.What's next? An alternative would be to:Flush the essential EXIF data immediately with a See more button,Once the page is loaded, do another JSON request which loads the remaining data without the user noticing something is happening in the background. Once the data is loaded, either See more can be replaced by the actual result, or may act as before, just without the 1 second delay.