Board index » cppbuilder » Re: Qt 4.4 Released

Re: Qt 4.4 Released


2008-05-12 10:53:12 PM
cppbuilder32
"Ed Mulroy [TeamB]" < XXXX@XXXXX.COM >writes:
[snip]
Where the C interface is clearly inferior is with generic code, and
non-fundamental types.
template <class T>
void f(T obj)
{
sprintf("???", obj);
}
Not only is it hard (but not impossible) to determine the proper
format specifier for types (though it is possible):
char const * format_str_for_type(int) { return "%d";}
char const * format_str_for_type(char) { return "%c";}
char const * format_str_for_type(float) { return "%f";}
char const * format_str_for_type(char const *) { return "%s";}
template <class T>
char const * format_str_for_type(T const &) {
std::ostringstream oss;
oss << "Unknown type - unable to return printf format str for: ";
oss << typeid(T).name();
throw oss.str();
}
etc.
template <class T>
void f(T obj)
{
sprintf(format_str_for_type(obj), obj);
}
There still is the problem with non-primitive types. You are not
allowed to pass C++ objects through functions taking ... parameters.
You are not allowed to extend the types of things that the C io
library handles. In short, you have to build a custom streamer on top
of the C syntax that is unrelated to the C syntax, for any
non-primitive types.
For safety, we would need some sort of meta-programming inside f() to
check if type T is a primitive type or not. If it is, THEN it can
call sprintf and use the format_str_for_type() helper. Otherwise, it
must do "something else".
--
Chris (TeamB);
 
 

Re:Re: Qt 4.4 Released

na wrote:
Quote
"Andre Kaufmann" < XXXX@XXXXX.COM >wrote in message news: XXXX@XXXXX.COM ...

>But I wouldn't use pure C++ for cross platform programming too.
>You would have to use the C++ standard library and IMHO the C++ iostreams library is quite unusable regarding
>performance.

Nothing *std::ios::sync_with_stdio(false);* does not fix...
Thank you. Although I don't understand why this should help in serially
writing to a file faster it's somewhat faster.
But still iostreams seem to be slower (in this simple example)
Andre
 

Re:Re: Qt 4.4 Released

Andre Kaufmann wrote:
Quote
>iostreams is a high level abstraction built on lower levels (and

That doesn't mean that it can't be fast ? And I don't think that it's
such a high level abstraction. A template using the << operator, which
writes a single variable to a file / buffer.
But abstractions tend to be generic.
And specific is usually faster than generic because
at some point, the generic must become specific.
Consider also, that in addition to wrapping the fwrite()
operation (slower than hand-coded), the << must decide
how to format the variable (sprintf). As you have seen,
the unformatted itoa is much faster.
You might further test the abstraction layer file speed by
writting fixed string "Data" instead of number conversions.
I suspect fwrite would be insignificantly faster than <<.
You could also try with/without strlen().
 

{smallsort}

Re:Re: Qt 4.4 Released

Andre Kaufmann wrote:
Quote
na wrote:
>"Andre Kaufmann" < XXXX@XXXXX.COM >wrote in
>message news: XXXX@XXXXX.COM ...
>
>>But I wouldn't use pure C++ for cross platform programming too.
>>You would have to use the C++ standard library and IMHO the C++
>>iostreams library is quite unusable regarding performance.
>
>Nothing *std::ios::sync_with_stdio(false);* does not fix...

Thank you. Although I don't understand why this should help in serially
writing to a file faster it's somewhat faster.
But still iostreams seem to be slower (in this simple example)
There is faster replacement
FASTreams
www.msobczak.com/prog/fastreams/
 

Re:Re: Qt 4.4 Released

Ed Mulroy [TeamB] wrote:
Quote
>Yes. Because it's not type safe. As you surely know.

"safe" and "type safe" are not the same thing. If something is not "safe",
it is a serious issue.
My opinion is that many (not all) C functions are unsafe and some
additionally are not type safe.
And please don't over interpret unsafe as "totally unsafe". I'm using
unsafe in the context of comparing C++ to C, meaning that the C++
functions / objects are safer to use as the C functions.
If I would generalize unsafe I couldn't use integers (generally) safely
either.
E.g.:
if (lenString1 + lenString2>maxStringLen)
might be unsafe too.
Quote
If something is not "type safe" the consequence is
that you must pay attention to what you are doing. Characterizing
that as
Quote
"inferior" is incorrect.
The problem is that you sometimes can't pay attention, without
implementing the functions on your own, because you can't specify the
length of the output buffers in many / perhaps all C functions using
output buffers.
Quote
>And the old C functional interfaces are the main reason for buffer
>overflows, because they are inherently unsafe.

A couple of functions have no ability to avoid such things. You do not
decry those functions. You damn the entire API.
The discussion was mainly about the file/string functions. I have only
argued that I think the C functions to be not safe enough, because you
can make many mistakes, which are simply not possible with their C++
counterparts.
So with "C functional interfaces" I meant file/string functions, not
generally the whole interface.
Quote
[...]
You created 'buffer' so are responsible its contents.
And if you are making mistakes and can't control the buffer or
formatting string ? How do you ensure that the fprintf function doesn't
overwrite the buffer boundaries. Where can you specify that the
resulting string doesn't overwrite the buffer boundaries.
E.g:
char format[] = "%s %d %f %s %d";
sprintf(buffer, format, s1, i1, f1, s2, i2);
Do you really expand all the arguments in [format] to check the
resulting length ?
Quote
If 'buffer' is
incorrect, that is due to a failure to correctly design and code and not a
No. fprintf, sprintf, printf is unsafe by design, since you can't
specify a maximum length for
Quote
consequence of which I/O routine was used. Examination and additional
handling of it by the '<<' call is redundant.
Short example. You have the string:
yyyyy: %s xxxxxx: %d in a resource file.
Which gets (false) translated in another language to:
rrrrr: %d zzzzzz: %s in a resource file.
O.k. the translator made the mistake, but how does the developer prevent
any unexpected crashes due to false format strings with sprintf ?
Regarding readability I would favor the C way over stringstreams, but
IMHO the best alternative would be to use the C# or Boost way:
cout << boost::format("writing %1%, x=%2% : %3%-th try")
% "hello" % 1 % 2;
Type safe and more overflow safe than printf and more readable than
stringstreams - same as printf.
Quote
Not the best example. Overflow danger arguments might better be attributed
to input than output.
Where does the input - the data I write to the file come from ?
Quote
If you rail against scanf, fscanf or gets then you have an argument, but one
which is against those specific functions and not against the whole C API.
The discussion was about file functions and perhaps additionally string
functions. I don't want to express all C functions to be generally
unsafe. But there are many of them which are unsafer as their C++
counterparts.
Quote
. Ed
Andre
 

Re:Re: Qt 4.4 Released

Your reply seems to be mis-aimed. It creates and attempts to give a C++
solution for a problem not mentioned. Perhaps you might post it again as a
reply to the message to which you were intending to reply.
. Ed
Quote
Chris Uzdavinis wrote in message
news: XXXX@XXXXX.COM ...

[snip]

Where the C interface is clearly inferior is with generic code, and
non-fundamental types.


template <class T>
void f(T obj)
{
sprintf("???", obj);
}

Not only is it hard (but not impossible) to determine the proper
format specifier for types (though it is possible):


char const * format_str_for_type(int) { return "%d";}
char const * format_str_for_type(char) { return "%c";}
char const * format_str_for_type(float) { return "%f";}
char const * format_str_for_type(char const *) { return "%s";}

template <class T>
char const * format_str_for_type(T const &) {
std::ostringstream oss;
oss << "Unknown type - unable to return printf format str for: ";
oss << typeid(T).name();
throw oss.str();
}

etc.

template <class T>
void f(T obj)
{
sprintf(format_str_for_type(obj), obj);
}


There still is the problem with non-primitive types. You are not
allowed to pass C++ objects through functions taking ... parameters.
You are not allowed to extend the types of things that the C io
library handles. In short, you have to build a custom streamer on top
of the C syntax that is unrelated to the C syntax, for any
non-primitive types.

For safety, we would need some sort of meta-programming inside f() to
check if type T is a primitive type or not. If it is, THEN it can
call sprintf and use the format_str_for_type() helper. Otherwise, it
must do "something else".
 

Re:Re: Qt 4.4 Released

Andre Kaufmann wrote:
Quote
>You created 'buffer' so are responsible its contents.
And if you are making mistakes and can't control the buffer or
formatting string ?
If you have no control, then you can't use a function
that requires control. Functions that don't require
control must be more complex (slower).
OTOH if you _do_ have control, then a function that
requires control can be much faster because it relies
on write-time checks instead of run-time checks.
Quote
How do you ensure that the fprintf function doesn't
overwrite the buffer boundaries. Where can you specify that the
resulting string doesn't overwrite the buffer boundaries.
By the use of precision.
char buffer[31];
sprintf( buffer, "%.30s", somecharptr );
Quote
char format[] = "%s %d %f %s %d";
sprintf(buffer, format, s1, i1, f1, s2, i2);

Do you really expand all the arguments in [format] to check the
resulting length ?
Normally, yes. I give each one a size/precision.
If it is just debug code, no, but then buffer
is 1k and s is known to be small.
Quote
Short example. You have the string:
yyyyy: %s xxxxxx: %d in a resource file.
Which gets (false) translated in another language to:
rrrrr: %d zzzzzz: %s in a resource file.

O.k. the translator made the mistake, but how does the developer prevent
any unexpected crashes due to false format strings with sprintf ?
If that is an issue, the programmer must either
Lock down the strings or
1) provide for rearranged data.
2) build the format string at runtime from a field table.
or
3) build the output string directly from the field table (easier).
Quote
>Not the best example. Overflow danger arguments might better be attributed
>to input than output.

Where does the input - the data I write to the file come from ?
Presumably from inside your application, after having been
read/checked/cleaned up from wherever it originated.
Quote
The discussion was about file functions and perhaps additionally string
functions. I don't want to express all C functions to be generally
unsafe. But there are many of them which are unsafer as their C++
counterparts.
Safety costs.
The question then is do you spend your safety dollars
up-front (development) or runtime (poor performance)?
 

Re:Re: Qt 4.4 Released

"Andre Kaufmann" < XXXX@XXXXX.COM >wrote in message
Quote
>na wrote:
>Nothing *std::ios::sync_with_stdio(false);* does not fix...
Thank you. Although I don't understand why this should help in serially writing to a file faster it's somewhat faster.
But still iostreams seem to be slower (in this simple example)
Glad it helped a little. By the way this is how I implement it. UNTESTED just a quick type up.
Does this still run too slow on your system?
****************************
std::ios::sync_with_stdio(false);
std::ofstream outfile;
outfile.open("C:\\Temp\\File.dat", std::ios::out | std::ios::binary);
char buf[1024];
while (true)
{
iBytesRead = pSocket->read(buf,1024);
outfile.write(buf, iBytesRead);
}
outfile.close();
****************************
 

Re:Re: Qt 4.4 Released

na wrote:
Quote
"Andre Kaufmann" < XXXX@XXXXX.COM >wrote in message

>>na wrote:
>>Nothing *std::ios::sync_with_stdio(false);* does not fix...

>Thank you. Although I don't understand why this should help in serially writing to a file faster it's somewhat faster.
>But still iostreams seem to be slower (in this simple example)

Glad it helped a little. By the way this is how I implement it. UNTESTED just a quick type up.
Does this still run too slow on your system?

****************************
std::ios::sync_with_stdio(false);
std::ofstream outfile;
outfile.open("C:\\Temp\\File.dat", std::ios::out | std::ios::binary);

char buf[1024];

while (true)
{
iBytesRead = pSocket->read(buf,1024);
outfile.write(buf, iBytesRead);
}

outfile.close();
****************************
What is missing here after opening file is this:
outfile.imbue(std::locale("C"));
By doing this we are making sure that no conversion is done whatsoever
on passing data.
This is very important especially if binary data are being processed.
 

Re:Re: Qt 4.4 Released

Bob Gonder wrote:
Quote
Andre Kaufmann wrote:
[...]
Consider also, that in addition to wrapping the fwrite()
operation (slower than hand-coded), the << must decide
how to format the variable (sprintf). As you have seen,
the unformatted itoa is much faster.
I haven't profiled why iostream is that slow, but I think it's mainly
because (in one of the implementations of iostream):
- exception handling
- sprintf is used in one implementation.
- some other overhead
If the implementation of itoa is faster, then it could be used instead
of sprintf - couldn't it ?
But deriving from other posts, I think there isn't that much interest in
why iostreams are that slow, it's simply seen as fact and critics aren't
welcome (that much).
Am I the only one, who thinks it to be ridiculous, that one
implementation is 5 times slower than the equivalent C# code ?
(I always thought code for a virtual machine to have much more overhead
than C++ code)
Side note: itoa could also be somewhat faster too, if the radix
parameter would be specialized.
Quote
[...]
Andre
 

Re:Re: Qt 4.4 Released

Bob Gonder wrote:
Quote
Andre Kaufmann wrote:

[...]
OTOH if you _do_ have control, then a function that
requires control can be much faster because it relies
on write-time checks instead of run-time checks.
O.K. But adding a simple output buffer length parameter to each print
function wouldn't be IMHO that much overhead.
Quote
>How do you ensure that the fprintf function doesn't
>overwrite the buffer boundaries. Where can you specify that the
>resulting string doesn't overwrite the buffer boundaries.

By the use of precision.
char buffer[31];
sprintf( buffer, "%.30s", somecharptr );
Good idea. But I meant to be able to specify the length of the buffer
for the sprintf function itself, as in the sprintf_s function (strsafe.h
- Windows SDK).
It's IMHO safer, if the print function checks by itself if the buffer
end has been reached - you can't simply forget to check it.
Quote
[...]
>O.k. the translator made the mistake, but how does the developer prevent
>any unexpected crashes due to false format strings with sprintf ?

If that is an issue, the programmer must either
Lock down the strings or
1) provide for rearranged data.
2) build the format string at runtime from a field table.
or
3) build the output string directly from the field table (easier).
Yes. This is what iostreams do ;-) - concatenating output strings.
But you could simple remove the type specifier and let the compiler do
it's task - calling the convert to string function for each parameter in
a type safe way - as the boost format function does.
E.g. instead:
"%s %d %d %d"
"%0% %1% %2% %3%"
%x% means insert here parameter number [x].
So the translator can even rearrange the format string and position of
the parameters. The output string can also be false, but since the
function returns a string no output buffer can be overwritten.
Quote
[...]
Safety costs.
The question then is do you spend your safety dollars
I doesn't cost that much, to use safe C string functions.
You simply could use strsafe.h from the Windows-SDK. The downside is
that it's not part of the C/C++ standard
and therefore not platform independent.
IIRC it has been rejected by the committee, but I don't know why.
Quote
up-front (development) or runtime (poor performance)?
Should I then use C#, because it's safe and 5 times faster ? No, I don't
state it to be generally faster as C++. It isn't.
But I haven't expected it to be that much faster in writing integers to
a text file.
Andre
 

Re:Re: Qt 4.4 Released

Darko Miletic wrote:
Quote
Andre Kaufmann wrote:
[...]
There is faster replacement

FASTreams
www.msobczak.com/prog/fastreams/

Thank you very much for the link.
Andre
 

Re:Re: Qt 4.4 Released

Ed Mulroy [TeamB] wrote:
Quote
Your reply seems to be mis-aimed. It creates and attempts to give a C++
I think it isn't mis-aimed.
Quote
solution for a problem not mentioned.
You stated the C interface not to be inferior ? Didn't you ?
May I cite:
'Characterizing that as "inferior" is incorrect.'
Quote
Perhaps you might post it again as a
reply to the message to which you were intending to reply.
[...]
Andre
 

Re:Re: Qt 4.4 Released

Andre Kaufmann wrote:
Quote
QT4 is a good framework, for sure. But the price of the commercial
version is>for me< too high, if the price I've found is correct.
But you don't need to buy components - because almost everything is there -,
you have the source code and a huge opensource ressource.
And it is easy to expand. And support is included !
T.
 

Re:Re: Qt 4.4 Released

Andre Kaufmann < XXXX@XXXXX.COM >writes:
Quote
I haven't profiled why iostream is that slow, but I think it's mainly
because (in one of the implementations of iostream):

- exception handling
- sprintf is used in one implementation.
- some other overhead
In one program I was working on, I made direct use of sprintf to
create a massive number of strings sent across the network.
(Representing every stock market data change for a single data
source.)
Profiling revealed that 52% of the entire runtime was spent inside
sprintf, and I only did things as complicated as:
sprintf(buf, "%d", val);
and similar for doubles. Since I wasn't using any of the fancy format
options, it was wasteful to use a function that parsed them every
single time (like sprintf). I hand-wrote an optimized int-to-string
function that was specific to my needs and simple format requirements,
and vastly reduced the amount of calls to sprintf. As a result, the
program spent less than 2% in sprintf's internals, and along with a
few other optimizations, throughput increased by 20,000 messages per
second.
(This was using g++'s standard library, not C++ Builder, so don't
extrapolate too much here.)
I am now of the position that the printf family of functions are not
so fast afterall, since you pay for the generality.
--
Chris (TeamB);