ContextI'm designing a database which, simplified, should be able to handle users sending job requests to each other, and after that a job can be started, finished, and reviewed. The design should be scalable (think millions of users).Approaches I've considered:Gargantuan tableOne approach, probably not the best one, would be to simply store ALL jobs in one, huge table jobs. This table would need a state column to represent which state the job is currently in (e.g. ACCEPTED, STARTED, FINISHED, REVIEWED e.t.c.). The biggest problem with this approach that I can see is that jobs in different states have different types of data that are relevant to them. For example, a job request has a preliminary agreed upon price, but that could change before the job is started, and change again before the job is finished. This could of course be solved by just adding more columns to the table and naming them properly, but it will probably become a huge bottleneck performance-wise very early to have one table containing all the different types of possible data for all the different possible states of a job.Different tables for different statesThis approach would be to have multiple tables, for example job_requests, jobs_started, jobs_finished, tables who in turn can have substates, e.g. job_requests could have the sub-states PENDING, ACCEPTED, while the jobs_finished table would have the substates COMPLETED, CANCELLED, REVIEWED.With this approach each table only contains data which is relevant to the current job state, but on the other hand some data might be duplicated (for example the user ids of the job requester, and job receiver -- on the other hand this information could be stored in yet another table?). The problem with this approach is that I can't think of a good solution on how to archive all the information when transitioning between states. For example, once a job request has been accepted, and then started, it should be deleted from the job_requests table and moved into the jobs_started table, but it's just a matter of time before a stakeholder wants to know for example how long the average time is between a job request being created, until it's been started, at which point I'd need the data from the job_requests table to be able to calculate it. It feels like this type of problem should be easy to solve, but I really can't think of any good solution which feels right, any solution I come up with feels ugly and I can immediately think of a number of things which makes the solution bad. Very grateful for any feedback or tips on approaches I could take. Thanks in advance!
Database design for objects with multiple states
architecture;database;database design;enterprise architecture;architectural patterns
Sounds like you have 3 major categories of data you are trying to store:General job data (job id, job requester id, job receiver id etc)State transitions (job started, job finished)State-specific job data(optional) job-related events (price changed, job receiver user reassigned etc)The key is to separate event-like data from everything else.Schema designHere are some details:1. General jobs tableAll of the information that's NOT state-specific goes into the (let's say) jobs table. Auto-generated primary key: job_id2. State transitions tableAll information about state transitions goes into job_state_transitions table, which might have the following columns:job_transition_idjob_idcreated_atfrom_stateto_stateIdeally this table is append-only. Nothing is every updated or removed here.Using such a table, you can find out the latest status of any particular job by selecting the latest row for a given job_id from the job_transitions table. You can further denormalize that and introduce a job_state column, the contents of which are updated every time a new row is inserted in job_transition table (stored procedures might help here if that's your thing).You can also do all sorts of analysis on state transitions, because the timing data is preserved (created_at is a date/time field that can help with that)3. State-specific job dataAll of the state-specific data goes into [state]-jobs tables. Primary key: some sequence id. Main index: job_transition_id4. Optional event dataYou can also introduce an audit trail table that allows you to keep track of various changes that users might request for each job, like your change agreed upon price. This is a generalization of the state transition table: one main table that contains events and a one supplemental table for each event type (for example, price_changes table with job_id, created_at, from_price and to_price columns).Scaling to millions of usersIf the main jobs table grows to be unwieldy, you can shard it by job_id or requesting_user_id or something like that.Likewise, events table should be append-only and can be rotated or purged of events related to jobs that have been finished.