SELF_TEST_DESCRIPTION index range

[tim_ecue]

Dear RDM friends,

I am somewhat unhappy with implementing the SELF_TEST_DESCRIPTION parameter.

The problem is, that in my understanding the current spec provides no indication of the potential index range for tests other than 1-254. It is not defined that the tests need to be consecutively ordered and there is no mechanism for determining the number of available tests.

Thus, it seems any fully complient implementation ends up enumerating all possible values. This requires 254 requests and takes about 9 seconds alone at 30 updates per second. This is really bad, particularly for large installations where you might need to poll hundreds of fixtures on a single DMX line, wich is unfortunately very realistic for me. Even for just 100 fixtures (thats just a normally populated one line of RGBW faders) this takes 15 minutes for fetching that damn thing alone. Customers hate it.

Eventually, I decided to simply default in the software to assume consecutively ordered numbers and allow an override for "deep" search, but I am not happy with it.

So after all that text: did I miss something or is it that bad ? Wouldnt it be useful to require consecutively ordered tests as for DMX personalities?

Best,
Tim

[hamish]

Hi Tim.

This part of RDM was not well thought out, if at all.
I've run in to this problem before, the biggest problem is finding a way to implement this in a generic controller, see the link for the previous discussion.
http://www.rdmprotocol.org/forums/showthread.php?t=1063
http://www.rdmprotocol.org/forums/showthread.php?t=24

My approach is as you describe, test every possibility to determine what tests are available.

Hamish

[tim_ecue]

Hi Hamish,

oops, I missed the other discussion. Thanks for the advice!

Did You already see a device where the test numbers where not consecutive ?

[hamish]

Tim.

Few of the devices I worked with or tested have any self tests. In answer to you question, no, not yet. I have however seen self tests from one manufacturer being different on different products, which in my estimation is not keeping with the spirit of (maybe also the requirement*) of the standard.

I would expect self tests to be treated in the same way as manufacturer specific PIDs, being universal across all products.

Hamish.

* I haven't checked before writing this.

[sblair]

There was no requirement that Product A and Product B from the same manufacturer have the same self-tests. The intent was always that the Self Tests be consecutively numbered within a product as discussed in the other thread that was referenced.

[tim_ecue]

Originally Posted by hamish

Tim.

Few of the devices I worked with or tested have any self tests. In answer to you question, no, not yet. I have however seen self tests from one manufacturer being different on different products, which in my estimation is not keeping with the spirit of (maybe also the requirement*) of the standard.

I would expect self tests to be treated in the same way as manufacturer specific PIDs, being universal across all products.

Hamish.

* I haven't checked before writing this.

Good info!

For me it feels like the standard just leaves this open.

What eventually counts is what You can rely on in the real world. There, I would not even dare to assume anything about different devices of the same manufacturer supporting the same manufacturer-specifc pid. Even there, I would not really dare to assume these are the same and I am not sure if this is explicitly stated in the spec (reusing Your * here )

So I overall doubt the usefulness of suggesting consistent manufacturer-specific test IDs because at first sight it seems that control software cannot reliably exploit these anyway. Consecutively numbered tests are at least simple and straightforward :D

[tim_ecue]

Originally Posted by sblair

There was no requirement that Product A and Product B from the same manufacturer have the same self-tests. The intent was always that the Self Tests be consecutively numbered within a product as discussed in the other thread that was referenced.

My favourite. Still, will it make it into the standard?

BTW: Yes, we have full E1.20 coverage in our upcoming sw release ... (or what I think full coverage is )

[hamish]

There was no requirement that Product A and Product B from the same manufacturer have the same self-tests. The intent was always that the Self Tests be consecutively numbered within a product as discussed in the other thread that was referenced.

Not withstanding the 'intent', the point I omitted to make is that (as Eric alluded to) this then makes a monkey of the self test vendor cast.

[sblair]

We can't really go back and change requirements on existing functionality. We can put a "should" statement in there but can not make it a "shall" because there are existing products in the field already.

[tim_ecue]

Originally Posted by sblair

We can't really go back and change requirements on existing functionality. We can put a "should" statement in there but can not make it a "shall" because there are existing products in the field already.

For sure true. But this essentially means that since the standard was unclear it could be either way, right?

I would still suggest addressing this issue in a future standard version. One example would be an optional parameter returning a bitmap of allocated IDs (PDLen=32)

Since we are on it I cannot resist throwing another one in: Similar issues arise when detecting subdevices. That could be addressed in a similar manner by returning a bitmap of present subdevices (PDLen=64).

[tim_ecue]

Originally Posted by sblair

We can't really go back and change requirements on existing functionality. We can put a "should" statement in there but can not make it a "shall" because there are existing products in the field already.

In this particular case it might also be that actually not mutch harm is done when clarifying this as originally intended.

It seems that very few (if any) actual responders currently implement non-consecutive IDs. In the worst case, a control software following the new standard will not recognize an available test on such a responder. Control software following the "scan all" approach is not affected anyway.

So if this in reality affects practically no one, it is indeed a common practice to clarify an existing standard in that way. HTTP for example has lots of such points.

Overall, it might be the easiest just to clarify that and still no one really gets hurt as the real world impact of this change might be very limited.

(oh my, people will start throwing things at me shortly ... )

[sblair]

Yes, we can add statements with "should" and "encourage" all day long unless there is someone during a Public Review cycle that screams bloody murder over it.

[tim_ecue]

Originally Posted by sblair

Yes, we can add statements with "should" and "encourage" all day long unless there is someone during a Public Review cycle that screams bloody murder over it.

"should" + advice that otherwise the test may not be detected by many control software systems would be sufficient.

Anything less than should + "otherwise it will not really work for your customers" is not strong enough.

I prefer "shall" given that no actual victims popped up so far. It is a strong statement and even if a few devices pop up, the number will be fairly limited and their IDs might even be explicitly published to enable control software vendors to add explicit support for these devices.

(Now I am already beyond bloody murder ... hehe)

[hamish]

(Now I am already beyond bloody murder ... hehe)

Hung for a sheep as a lamb.

I concur.

Now where is the rope?

[ericthegeek]

Originally Posted by tim_ecue

I prefer "shall" given that no actual victims popped up so far. It is a strong statement and even if a few devices pop up, the number will be fairly limited

Realistically we can't use that as an argument. We have to assume that there are RDM products we've never even heard of in far off lands whose language we can't speak and who've implemented RDM totally independently. They've never talked to a task group member, and probably don't even know the task group exists.

It's tempting to say "We don't know about anyone who does this, so we can change it without consequence". But we have to assume anything that can be done has been done.

[tim_ecue]

Originally Posted by ericthegeek

Realistically we can't use that as an argument. We have to assume that there are RDM products we've never even heard of in far off lands whose language we can't speak and who've implemented RDM totally independently. They've never talked to a task group member, and probably don't even know the task group exists.

It's tempting to say "We don't know about anyone who does this, so we can change it without consequence". But we have to assume anything that can be done has been done.

Hi Eric,

In general I would fully agree as standards need to be trustworthy. Still, this is a clarification, not a change. Furthermore, for this particular one I see little harm done for existing devices, but a great potential to fix a very critical issue. In the worst case, the control software will miss a test name. Still, since the parameter is not mandatory, it should anyway offer a way to trigger the test directly by number, so it should be still available for the user.

I am not sure if You truly recognize how bad this one really is. You have to keep in mind that we usually deal with very large installations that may have thousands to hundreds of thousands RDM fixtures. In practice this particular issue typically reduces performance (at least in the initial full examination of the fixture, which is crucial) for a fixture supporting that parameter to about 10% of what it would be if the numbers where consecutively ordered.

Since even without this catastrophic scenario slow performance is the major complaint customers have about RDM, this problem just breaks its neck because it makes it worse by a full order of magnitude. Our customers in the past have made it very clear, that poor RDM performance is a major reason for still using and even developing proprietary technology in place of RDM despite RDM being already available to them!

Best,
Tim

[ericthegeek]

All of the optimizations that you suggest are perfectly valid for an individual controller to do. I'm just uncomfortable mandating them as part of the standard.

If you're in a truly performance critical environment, you can cache the list between sessions. When the user brings up the self-test menu for a specific device, you can present them the cached list. While they're reading through the list, you can be querying the device to see if anything's changed in the background.

Further, I'd say it's reasonably safe to assume that responders with the same Model ID, Software Version ID, and Personality have the same list of self tests. Thus, if you have 100 responders of two different models, you can send a GET DEVICE_INFO to all of them, then query all 255 self-tests from one of each model. That's a total of 610 packets for 100 devices. It also scales well, because adding additional responders only requires one more DEVICE_INFO request.

If you're not in the middle of an active crossfade, you can drop back to one null startcode packet per second. It's perfectly reasonable to send 200 RDM requests a second if you're working with well-behaved responders. In a 4 universe system, you can do 800 requests a second. Thus, you could complete the query described above in about 3 seconds or less.

Architectural environments tend to change relatively infrequently. That gives lots of opportunities to cache data in the controller. Even if the initial setup takes a few minutes (even hours), subsequently a single GET DEVICE_INFO is enough to tell if the device is present, and if it's had new software installed.