Here is my exact scenario.I must make requests to a third party serviceThe service takes ~15 seconds to respondIt also has no webhooks or any call back mechanismsThese requests are not made constantly throughout the day, but in chunksThe requests to the third party service are made from a consumer that is pulling from a queueusing mass transit default impl & rabbitmqAll of the messages the product generates are currently put into one queue. The same queue with the calls to the third party serviceThere are currently 3 consumers, running 4 worker threads each for a total of 12 worker threads. This is the current maximum number of concurrent messages that can be handled at onceThroughout the day the message queue will receive ~100 of these messages that will back up the queue. All 12 worker threads will process the ~15s calls and the queue will begin to backup. This results in extreme loss in functionality. Delayed emails, etc.  It normally continues to grow the queue for ~30 minutes until it catches upWhen this occurs the consumers are running at 0% CPU because they are all in thread waits.The question I have is how do I setup my message queue and consumers so that they don't backup.  I am trying to prevent loss of functionality of the site for those 30 minute time blocks.  Anything goes.Recap:1 queue~15 seconds tasks running at 0% cpu12 consumer worker threadsQueue backs up and takes ~30 minutes to catch upusing mass transit default impl & rabbitmq
How do I handle long running third party calls from backing up my message queue?
performance;message queue