Thursday, 15 August 2013

How to architect an efficient batch processes for web application?

How to architect an efficient batch processes for web application?

I have a web application written in PHP using a Postgres database. The
next phase of development is for batch processes to be built that will
need to be executed once a day (or adhoc as requested) for each user of
the app. The process will query, await response and process the response
from third-party services to feed information into the user's account
within the web application.
Are there good ways to do this?
How would batches be triggered every day at 3am for each user?
Given there could be a delay in the response, is this a good scenario to
use something like node.js?
Is it best to have the output of the batch process directly update the web
application's database with the appropriate data? Or, is there some other
way to handle the output?

No comments:

Post a Comment