Choosing Partition and Sort Key for the DynamoDB for a job queue


DynamoDB newbie question

I am trying to build a job table which really acts as a priority queue(One Lambda fills once a day and other lambda executes by reading one at a time). Below is the high level flow:

  • First Lambda --> reads the job parameters from JSONs and inserts to DDB table --> Second Lambda --> reads each job and executes the job accordingly.

Trying to see what Partition Key and Sort Key would make sense? Couple of things I thought of

  1. UUID as JobID: But Second Lambda wouldn't know this random ID to read from the DDB Table
  2. IndexName#RunDate: For any reason, if the first lambda re-runs after few hours, records shall be overridden and the previous state is lost.
  3. IndexName#RunDate#Run1: This needs that first lambda to read the records to check if the records are already present and then increment the number

I am looking for some simpler approach.

질문됨 3달 전116회 조회
1개 답변

You mentioned priority but I'm not sure I understand the requirements around that. Are all the jobs for a particular day processed with the same priority? Or do you want to have them processed in a preferred order? Maybe you meant that the priority is just by day (so records from the earlier date get processed first)?

I think your first thought is likely to be the right start - each job has its own unique JobID in a table that has a simple key (partition key only - no sort key). And you will likely need a GSI (or two) from there. Just need a bit more of the priority processing detail I mentioned above to give recommendations on the GSI(s).

답변함 3달 전
profile picture
검토됨 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠