Interpretation of NR_BUFFER_FULL

[Thomas]

Hi all,

imagine the following situation:

The message queue of an RDM responder is full. A Set request arrives.
Saving needs too long to be answered with ACK. Therefore the answer is NACK with reason NR_BUFFER_FULL.

What behavior would you expect from the RDM responder?

1. The value is saved but no message written to the queue.
Because: If the RDM controller does not support QUEUED_MESSAGE it is still be able to configure the RDM responder also if the message queue is full. The successful saving can be checked using the corresponding Get request.

2. The value is not changed
Because: The responder reported NACK, which is defined as "the responder is ... unable to process the specified SET command"

3. Any other behavior ???

Best regards
Thomas

[nomis52]

I'd expect the value not to be changed, and the controller (or user) to retry later.

RDM controllers *have* to support queued messages if they are to work with proxies and any responder which uses ACK_TIMER.

[ericthegeek]

To me, a NR_BUFFER_FULL response means that the responder could not handle the request and therefore dropped it without acting on it. This is most similar to what you described as behavior #2.

But: The better answer is to design the responder in such a way that this condition can never occur. A well designed responder will be able to handle every possible queued message and thus never be subject to the kind of message queue overflow that you describe.

Traditional software queues tend to take a lot of memory (doubly linked lists, etc), but despite it being called "Queued Messages", a responder typically doesn't need to implement it as a queue. All you need is a bitmask with one bit for each possible PID that can result in a queued message indicating whether you need to send that PID in response to "GET QUEUED_MESSAGE". That's typically only a handful of bits in RAM. I also recommend you sent any Queued Messages that were triggered by an ACK_TIMER before sending any message that are due to internal events, but even at the worst case this is only 2 bits per PID.

[Thomas]

Thank you both for the answers!

Eric
do I get you right that only the last message per PID + SD + CC needs to be available in the message queue?

This would result in that a queue that has the size of (nPID * nSD * 2) never reports NR_BUFFER_FULL except if status messages are not supported. Would this be according to the standard?
I don't find a hint that allows to replace messages in the queue.

[ericthegeek]

Originally Posted by Thomas

do I get you right that only the last message per PID + SD + CC needs to be available in the message queue?

Yes, that is my opinion.

Consider the following sequence:
Using the menu interface on the Fixture itself, a user changes the DMX address of a fixture to 300.

The fixture adds a "GET_RESPONSE DMX_START_ADDRESS 300" to the message queue.

Then, 15 minutes later (having never retrieved the queued message), the console does a "SET DMX_START_ADDRESS 301" followed by a "GET DMX_START_ADDRESS"

Because it's still busy processing the SET, the fixture ACK_TIMERs the GET. When it's ready, it adds "GET_RESPONSE DMX_START_ADDRESS 301" to the queue.

Herein lies the problem. You now have two different "GET_RESPONSE DMX_START_ADDRESS" messages in the queue. One for 300 and one for 301.

Since the console received an ACK_TIMER, it now does a "GET QUEUED_MESSAGE". If the responder has implemented a true queue, it receives a "GET_RESPONSE DMX_START_ADDRESS 300". This data is 15 minutes out of date.

If the responder implemented a true queue, any time you did a GET QUEUED_MESSAGE you'd have to completely drain the queue of all messages before you could act on any of the messages because there *might* be newer data waiting later in the queue. On a busy responder that's generating lots of queued messages you may never be able to drain the queue.

Thus, it's my opinion that responder should always return the most current data in response to a GET QUEUED_MESSAGE. Also, if the same message has been added to the queue multiple times, it only needs to be returned once in response to a GET QUEUED_MESSAGE until the data changes again.

Originally Posted by Thomas

This would result in that a queue that has the size of (nPID * nSD * 2) never reports NR_BUFFER_FULL except if status messages are not supported. Would this be according to the standard?

Status Messages and Queued Messages behave differently. What I've said applies to Queued Messages. Some of the same concerns may apply to Status Messages as well. Frankly, I don't think the best practices for status messages have been fully worked out yet, especially since the addition of STATUS_CLEARED in E1.20-2010.

Originally Posted by Thomas

I don't find a hint that allows to replace messages in the queue.

There's no text directly supporting this in the standard. It's entirely my opinion of best practices for how queued messages should work based on operational experience.

[nomis52]

Originally Posted by ericthegeek

Since the console received an ACK_TIMER, it now does a "GET QUEUED_MESSAGE". If the responder has implemented a true queue, it receives a "GET_RESPONSE DMX_START_ADDRESS 300". This data is 15 minutes out of date.

On the other hand, the controller would also see a non-0 message count, which indicates that there is more messages in the queue.

Originally Posted by ericthegeek

If the responder implemented a true queue, any time you did a GET QUEUED_MESSAGE you'd have to completely drain the queue of all messages before you could act on any of the messages because there *might* be newer data waiting later in the queue. On a busy responder that's generating lots of queued messages you may never be able to drain the queue.

The problem is though that if there are proxies in the network you do see FIFO queuing behavior. This is the reason why the RDM tests drain all queued messages prior to each test.


Simon (who prefers the real queue method and NR_BUFFER_FULL responses)

[sblair]

I agree with the best practices Eric mentioned. My implementations have always pretty much reflected what he posted so that I avoid returning stale data.