Board index » cppbuilder » Looking for advice

Looking for advice


2003-09-03 01:51:10 AM
cppbuilder56
OK, I'm breaking newsgroup rules in that I posted something similar to
Java non-tech, but traffic there is slower and I don't know the names to
lend trust to opinions. I know the people round here and need an answer
with experience I can trust <g>
We have just been given a requirement from our governing body that they
want a real-time feed of certain data from our car at 10Hz. This will
have to be delivered whenever the car is on track, so the worst case
scenario is during a race which can be up to 2 hours.
We currently have a Java library that can give us this information, but
I have no experience of Java in a real time environment. Will it be
safe to rely on Java to serve this data at 10Hz uninterupted for 2
hours? How is garbage collection going to affect us?
We need to deliver within 4 weeks, and I don't want to be responsible
for a race disqualification because our telemetry feed timed out!
Migrating to C++ would give me a familiar environment, but would brings
enough problems that I would need strong evidence to justify the
decision, rather than just my paranioa and FUD <g>
I simply lack experience to make the call. Deadlines are tight, the
Java solution allows us to use more of our existing libraries so is
preferred if it is appropriate, but we can't afford to get the QoI
wrong. I need to hear from experience I can trust!
Anyone out there with experience for/against Java in this regard?
--
AlisdairM
Team Thai Kingdom
 
 

Re:Looking for advice

"Alisdair Meredith" < XXXX@XXXXX.COM >schrieb im
Newsbeitrag news: XXXX@XXXXX.COM ...
Quote
We have just been given a requirement from our governing body that they
want a real-time feed of certain data from our car at 10Hz. This will
have to be delivered whenever the car is on track, so the worst case
scenario is during a race which can be up to 2 hours.
this would mean 7.200 data transmissions within 2 hours - how much data will
be exchanged ?
Quote
We currently have a Java library that can give us this information, but
I have no experience of Java in a real time environment. Will it be
safe to rely on Java to serve this data at 10Hz uninterupted for 2
hours? How is garbage collection going to affect us?
How is the Java library accessing the incoming data ? sockets ? JNI ?...
depends on JVM you use - using 1.4 would give you better results than 1.3 !
experimenting with some parameters for the JVM could give you more security
/ robustness...
IIRC there are even JVM switches to influence automatic garbage
collection...
About memory usage I would put the code on a dual-CPU machine with at least
twice the RAM of maximum data you will be receiving...
even if garbage collection would do some harm - I expect the Java code to be
still responsive as long as availabe resources are much more than needed..
Another point :
there are some tools to stress test Java code in the areas memory usage and
multi-threading issues...
I would do some sort of stress test on this Java library...
Quote
We need to deliver within 4 weeks, and I don't want to be responsible
for a race disqualification because our telemetry feed timed out!
Migrating to C++ would give me a familiar environment, but would brings
enough problems that I would need strong evidence to justify the
decision, rather than just my paranioa and FUD <g>
I simply lack experience to make the call. Deadlines are tight, the
Java solution allows us to use more of our existing libraries so is
preferred if it is appropriate, but we can't afford to get the QoI
wrong. I need to hear from experience I can trust!

Anyone out there with experience for/against Java in this regard?
I know of Java apps doing something similar pretty fine but they have been
finetuned to make it as secure as possible...
I don't what work would be involved in doing a simulation...
I would put the Java solution under hard conditions ( perhaps harder than
reality - let's say data at 20Hz and 4 hour time range ) and see what
happens...
If you need realtime constraints - meaning that response times have to be
within exactly defined timeframes then you will have to go for some realtime
capable base components :
- there are options running on Win2K ( and needing usually C / C++ )
- there are JVMs out there modified to meet realtime needs
- there are some RealTimeOS out there ( some being based on good old DOS,
some on Linux... )
My 2 cents...
Yahia
 

Re:Looking for advice

Alisdair Meredith wrote:
Quote
OK, I'm breaking newsgroup rules in that I posted something similar to
Java non-tech, but traffic there is slower and I don't know the names to
lend trust to opinions. I know the people round here and need an answer
with experience I can trust <g>

We have just been given a requirement from our governing body that they
want a real-time feed of certain data from our car at 10Hz. This will
have to be delivered whenever the car is on track, so the worst case
scenario is during a race which can be up to 2 hours.

We currently have a Java library that can give us this information, but
Where would this run? On the car (seems inlikely)? Or the PC?
Quote
I have no experience of Java in a real time environment. Will it be
safe to rely on Java to serve this data at 10Hz uninterupted for 2
hours? How is garbage collection going to affect us?

I have a first-hand experience with a Java GC interfering with a
real-time environment. This is in the VoIP world.
How complicated is the thing the library is doing?
.a
p.s. no more brake checks DC, please <g>
 

{smallsort}

Re:Looking for advice

Alisdair Meredith < XXXX@XXXXX.COM >writes:
Quote
OK, I'm breaking newsgroup rules in that I posted something similar to
Java non-tech, but traffic there is slower and I don't know the names to
lend trust to opinions. I know the people round here and need an answer
with experience I can trust <g>
Don't trust me! <g>
Quote
We have just been given a requirement from our governing body that they
want a real-time feed of certain data from our car at 10Hz. This will
have to be delivered whenever the car is on track, so the worst case
scenario is during a race which can be up to 2 hours.

We currently have a Java library that can give us this information, but
I have no experience of Java in a real time environment. Will it be
safe to rely on Java to serve this data at 10Hz uninterupted for 2
hours? How is garbage collection going to affect us?
Alisdair, I sincerely hope that your "governing body" specifies things
better than you do here! :-)
What Java VM will you use?
Which OS?
Machine specifications?
10 Hz. Ok. But how much data? What kind of transmission media or
interface? Is the transmission very reliable (that is, how frequently
you need to re-transmit)
How accurate must be starting time of the send?
... and so on.
The only reliable response to your question is given by the
experience. Simulate the whole thing, as accurate as possible. Vary
the experiment parameters and see how that affects the results. Don't
expect that running the thing at a higher frequency will give you more
reliable results! The JVM or the OS does some things (i.e. garbage
collection) depending on the work load.
Ask the JVM authors. Ask the library authors. Experiment. Then, you
will sleep easily or not :-)
Quote
We need to deliver within 4 weeks, and I don't want to be responsible
for a race disqualification because our telemetry feed timed out!
If something like that happens to Fernando, dare coming to Spain!
Quote
Migrating to C++ would give me a familiar environment, but would brings
enough problems that I would need strong evidence to justify the
decision, rather than just my paranioa and FUD <g>

I simply lack experience to make the call. Deadlines are tight, the
Java solution allows us to use more of our existing libraries so is
preferred if it is appropriate, but we can't afford to get the QoI
wrong. I need to hear from experience I can trust!
You are on a though position. How much confidence do you have on the
library authors? More than on yourself for solving the problem with
C++?
Quote
Anyone out there with experience for/against Java in this regard?
Java was designed with embedded realtime devices on mind (or so Sun
says). How well it performs on this regard for a given device, depends
on the JVM and on the software.
Realtime applications are really delicate. Take care.
--
Oscar
 

Re:Looking for advice

Yahia El-Qasem wrote:
Quote
this would mean 7.200 data transmissions within 2 hours - how much data will
be exchanged ?
Data loads are small, each transmission will the < 1K <g>It is the QoI
constraints that I am worried about.
Quote
How is the Java library accessing the incoming data ? sockets ? JNI ?...
HTTP!
Quote
depends on JVM you use - using 1.4 would give you better results than 1.3 !
experimenting with some parameters for the JVM could give you more security
/ robustness...
IIRC there are even JVM switches to influence automatic garbage
collection...
Cool, our library 'migrated' to 1.4 just two weeks back. Looks like my
luck is in <g>
Quote
About memory usage I would put the code on a dual-CPU machine with at least
twice the RAM of maximum data you will be receiving...
even if garbage collection would do some harm - I expect the Java code to be
still responsive as long as availabe resources are much more than needed..
Unfortunately I have no control over the PC we will run on, it will like
be the least busy PC at the track. This means it is likely to be busy
doing other things, but probably not during the times we have to serve
data.
Quote
Another point :
there are some tools to stress test Java code in the areas memory usage and
multi-threading issues...
I would do some sort of stress test on this Java library...
Yes, we will certainly try to test this and simulate conditions before
going live. I was just double-checking before starting the work that
the QoI is an attainable target with current Java technologies. We
don't really have time for another go if testing proves we need to start
again with a different set of tools!
Quote
If you need realtime constraints - meaning that response times have to be
within exactly defined timeframes then you will have to go for some realtime
capable base components :

- there are options running on Win2K ( and needing usually C / C++ )
- there are JVMs out there modified to meet realtime needs
- there are some RealTimeOS out there ( some being based on good old DOS,
some on Linux... )

My 2 cents...
Thanks, well worth the asking price ;?)
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

"Alex Bakaev [TeamB]" wrote:
Quote
Where would this run? On the car (seems inlikely)? Or the PC?
This is running on a PC in our pit-lane network.
Quote
I have a first-hand experience with a Java GC interfering with a
real-time environment. This is in the VoIP world.
That is exactly the experience I am looking for!
How big was the interference? What JDK were you using at the time?
Quote
How complicated is the thing the library is doing?
My client will be really simple, the library collects data from our
car's real time car-to-pit telemetry signal so I'm unlikely to rewrite
that in 4 weeks!
Quote
p.s. no more brake checks DC, please ?g?
When you're 30 seconds in the lead, 'brake checks' don't seem to have
the same effect ;?)
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

How does your radio link cope with tunnels at monaco? Can it keep up 10Hz
there?
Rgds Pete
"Alisdair Meredith" < XXXX@XXXXX.COM >wrote in message
"Alex Bakaev [TeamB]" wrote:
Quote
Where would this run? On the car (seems inlikely)? Or the PC?
This is running on a PC in our pit-lane network.
Quote
I have a first-hand experience with a Java GC interfering with a
real-time environment. This is in the VoIP world.
That is exactly the experience I am looking for!
How big was the interference? What JDK were you using at the time?
Quote
How complicated is the thing the library is doing?
My client will be really simple, the library collects data from our
car's real time car-to-pit telemetry signal so I'm unlikely to rewrite
that in 4 weeks!
Quote
p.s. no more brake checks DC, please ?g?
When you're 30 seconds in the lead, 'brake checks' don't seem to have
the same effect ;?)
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

Oscar Fuentes wrote:
Quote
Don't trust me! <g>
Too late!
Quote
Alisdair, I sincerely hope that your "governing body" specifies things
better than you do here! :-)
You would hope so wouldn't you <g>
We are due some specs later this week. For now you have as much info as
I have!
Quote
What Java VM will you use?
Sun 1.4
Quote
Which OS?
Win2K (OK, maybe this should be a bigger worry than Java?)
Quote
Machine specifications?
Don't know what will be available yet. Will be reusing hardware that is
already busy doing something else already, but hopefully it will not be
busy while the car is on the track.
Quote
10 Hz. Ok. But how much data? What kind of transmission media or
Data size is really not a concern, each packet will be less than one k.
Quote
interface? Is the transmission very reliable (that is, how frequently
you need to re-transmit)
How accurate must be starting time of the send?
... and so on.
All we currently have is a request to transmit certain information at
10Hz.
Quote
If something like that happens to Fernando, dare coming to Spain!
<g>
Quote
You are on a though position. How much confidence do you have on the
library authors? More than on yourself for solving the problem with
C++?
Library authors are the other half of our software department in
France. I trust them to do a good job. The main problem we have is
that they are Java zealots, but I am a C++ zealot. It is hard to solve
religious differences <g>
Because of our religious leanings, I simply don't trust them when they
tell me GC will not be an issue. This is probably my failing, but
unless they go and run the timings themselves I cannot find it in myself
to trust their assurances in this regard, just as they will not trust me
that using RAII techniques in C++ means I really don't need to worry too
much about memory leaks. That's why I need an impartial 3rd party I can
trust <g>
Quote
Java was designed with embedded realtime devices on mind (or so Sun
says). How well it performs on this regard for a given device, depends
on the JVM and on the software.

Realtime applications are really delicate. Take care.
Thanks.
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

Pete Fraser wrote:
Quote
How does your radio link cope with tunnels at monaco? Can it keep up 10Hz
there?
Good question! <g>
I beleive that while I may be getting signal dropouts, I will still have
to serve up 'dropout' responses at 10Hz. While the car may be in the
tunnel, our pitlane doesn't move!
However, Monaco has many more problems than just the tunnel. Eg the
pitlane and pitwall are a significant distance from each other, and so
typically not connected. Luckily, this should not be MY problem <g>
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

Quote
We have just been given a requirement from our governing body that they
want a real-time feed of certain data from our car at 10Hz. This will
have to be delivered whenever the car is on track, so the worst case
scenario is during a race which can be up to 2 hours.

We currently have a Java library that can give us this information, but
I have no experience of Java in a real time environment. Will it be
safe to rely on Java to serve this data at 10Hz uninterupted for 2
hours? How is garbage collection going to affect us?

rather than just my paranioa and FUD <g>

I simply lack experience to make the call. Deadlines are tight, the
Java solution allows us to use more of our existing libraries so is
preferred if it is appropriate, but we can't afford to get the QoI
wrong. I need to hear from experience I can trust!

First, how close to 10Hz does the timebase need to be?
How will you establish an accurate timebase (presumably to sample your
existing data stream?)
What is the data quantity per sample time?
What hardware and software platform do you have available on the car system
to execute the code?
and lastly, just out of curiousity, what is the data being sampled and
transmitted?
I'd say the above needs to be answered before you can ask whether or not
Java is suitable.
keith
 

Re:Looking for advice

Alisdair Meredith < XXXX@XXXXX.COM >writes:
[snip]
Quote
>Which OS?

Win2K (OK, maybe this should be a bigger worry than Java?)
Win2K is not a real-time OS. You can argue that 10Hz is not an extreme
request, and if the hardware and drivers are known to work ok I'll
agree with you. The question is *when* the process should start. 10Hz
just means 10 times per second. It's very different to request a
packet every 100ms +- 5ms that every 100ms +- 75ms. For the first
case, if a failure means a disqualification, I wouldn't trust Win2k
and I would go with a RT OS. Just my opinion. Anyways, as that machine
must attend at least two channels of communication (input data, output
data), be sure a blocking on the input data doesn't prevent the output
to be dispatched. I've seen that problem caused not only by bugs on
user software, but by problems with Win2K and the drivers of the
communication devices. One simulation I would do is an input channel
freeze (try it on the middle of an input packet ;-).
There are other possible issues, although less probable. I'll not
cause you more worry than you have now. Ignorance is the best relaxant
:-)
[snip]
Quote
All we currently have is a request to transmit certain information
at 10Hz.
Some folk is trying to make your life interesting. He decided that is
not fair to have a F1 pilot with so much stress while other people on
the team is just sitting on a chair, drinking lemonade <g>
[snip]
--
Oscar
 

Re:Looking for advice

Alisdair Meredith wrote:
Quote
"Alex Bakaev [TeamB]" wrote:


>Where would this run? On the car (seems inlikely)? Or the PC?


This is running on a PC in our pit-lane network.



>I have a first-hand experience with a Java GC interfering with a
>real-time environment. This is in the VoIP world.


That is exactly the experience I am looking for!
How big was the interference? What JDK were you using at the time?

We had slowdowns on the order of 5-7 seconds. This resulted in the
dropped calls. The SW run on a Sun box (not too powerfull), don't know
exactly what version of Java.
Quote

>How complicated is the thing the library is doing?


My client will be really simple, the library collects data from our
car's real time car-to-pit telemetry signal so I'm unlikely to rewrite
that in 4 weeks!

I'm confused at this point - the library already does what you want it
to do, but you have doubts about how applicable it is?
Quote

>p.s. no more brake checks DC, please ?g?


When you're 30 seconds in the lead, 'brake checks' don't seem to have
the same effect ;?)

At least one of them did <g>
.a
 

Re:Looking for advice

"Alex Bakaev [TeamB]" wrote:
Quote
I'm confused at this point - the library already does what you want it
to do, but you have doubts about how applicable it is?
Kind of. We have a Java library that connects to our real-time server.
I have used this library in the past to monitor the telemetry and raise
alarms when 'odd' things need attention. However, the requirements for
this were not too sensitive to dropouts.
Now I have what looks like much stricter QoI requirements so I want to
be sure that I can press this library into service again, or whether we
need a new C++ interface to the server.
As it turns out, there is already a C++ interface that has escaped our
attention due to the 'Java team's enthusiasm for all things Java. So it
is available but maybe untested.
The Java library is going to be the easiest solution to implement, but
we lack experience if anything out of the ordinary crops up.
"Play it safe" says use C++, as that is where our experience lies.
For once, "do it right" seems to suggest Java, as that is where we get
best support. The problem is being *so* reliant on that support.
I got a couple of responses on the JBuilder group as well, including a
couple of familiar faces <g>I am tempted to try the Java route while
turning on the 'incremental garbage collection' option in the JVM. From
the description it sounds like it should resolve the GC issue, or at
least reduce it to an acceptable risk.
I guess I'll know better once I have an implementation in hand to test
<g>
--
AlisdairM
Team Thai Kingdom
 

Re:Looking for advice

Alisdair Meredith wrote:
Quote
"Alex Bakaev [TeamB]" wrote:


>I'm confused at this point - the library already does what you want it
>to do, but you have doubts about how applicable it is?


Kind of. We have a Java library that connects to our real-time server.
I have used this library in the past to monitor the telemetry and raise
alarms when 'odd' things need attention. However, the requirements for
this were not too sensitive to dropouts.

I see.
Quote
I got a couple of responses on the JBuilder group as well, including a
couple of familiar faces <g>I am tempted to try the Java route while
turning on the 'incremental garbage collection' option in the JVM. From
the description it sounds like it should resolve the GC issue, or at
least reduce it to an acceptable risk.

I guess I'll know better once I have an implementation in hand to test
<g>

You can always do GC explicitly, safe to do so, so it will never have to
kick in automatically, throwing you off.
.a
 

Re:Looking for advice

"Alisdair Meredith" < XXXX@XXXXX.COM >schrieb im
Newsbeitrag news: XXXX@XXXXX.COM ...
Quote
Yahia El-Qasem wrote:

>this would mean 7.200 data transmissions within 2 hours - how much data
will
>be exchanged ?

Data loads are small, each transmission will the < 1K <g>It is the QoI
constraints that I am worried about.
I agree... means less than 10 MB within the two hours...
Quote
>How is the Java library accessing the incoming data ? sockets ? JNI ?...

HTTP!
HTTP is fine... and is very good doable in C++ ( for example with Indy )...
Quote
>About memory usage I would put the code on a dual-CPU machine with at
least
>twice the RAM of maximum data you will be receiving...
>even if garbage collection would do some harm - I expect the Java code
to be
>still responsive as long as availabe resources are much more than
needed..

Unfortunately I have no control over the PC we will run on, it will like
be the least busy PC at the track. This means it is likely to be busy
doing other things, but probably not during the times we have to serve
data.
As long as you don't have control over the PC your module will run on then I
would see exactly one way ( sorry to be extreme ) :
deliver a thoroughly tested C++ module ( no DLL dependancies etc. ) to run
on the "unknown" PC.
Background :
memory usage with C++ is very fine controllable - data processing
performance would be much nearer to the optimum...
for HTTP use a blocking library ( like Indy ) within a seperate thread...
I would even preallocate memory for all incoming data - as it will be less
than 10 MB this should be no issue... would allow a multi-threading design
with very little performance loss...
This is an extreme suggestion I know but I would say : for extreme
circumstances ( unknown PC, risk of disqualification... ) deliver the best
you can...
HTH
Yahia