Board index » delphi » Memory mapped files in win32 free pascal

Memory mapped files in win32 free pascal

I have just recently started doing some work in win32 - my previous
programming has been with Unix.

I want to share memory between two processes and use free Pascal to do
it.

I first read some old documentation that led me to believe that DLLs
would be the answer - they are required, but the code is copied into
each process that rather defeats the object of a shared library.

So, I now realise that I need to use memory mapped files,
createfilemapping, openfilemapping and mapviewoffileex - so that I can
map the file (actually the pagefile) to my variable space and have the
inter-process data sharing that I need. The only example I can find is
in c, on the ms site.

Surely people are doing this in free Pascal all the time? Does anybody
have some sample code that simply arranges for the global variables in
the DLL to be available as shared memory to all processes using the DLL?

If anybody has an example, or any better suggestions, please let me
know.

--
Great men are almost always bad men.
Lord Acton

 

Re:Memory mapped files in win32 free pascal


Quote
> Surely people are doing this in free Pascal all the time? Does anybody
> have some sample code that simply arranges for the global variables in
> the DLL to be available as shared memory to all processes using the DLL?

hFile=A valid file handle
SizeToMap=Size of data that should be mapped

hMap:=CreateFileMapping(hFile,nil,PAGE_READWRITE,0,SizeToMap,'MyFileMappingObject');

Each DLL that should access this shared memory do following in initialization:

Base=pointer to suggested memory address where the data will be mapped
into current address space.

hNewMap:=OpenFileMapping(FILE_MAP_ALL_ACCESS,false,'MyFileMappingObject');
MapViewOfFileEx(hNewMap,FILE_MAP_ALL_ACCESS,0,OffsetInFileToMap,NumberOfBytesToMap,Base);

Then add folowing variable export:

exports
  Base name 'MySharedMemory';

So, each process that loads this DLL can access this shared memory by DLL export
'MySharedMemory'.
There is no possibility to import variables in Delphi, there you should use
GetProcAddress(0,'MySharedMemory');
In Free pascal it should be possible to declare a import variable:

var
  MySharedMemory:pointer;external 'My.Dll' name 'MySharedMemory';

Accessing shared data is done by PMyType(MySharedMemory)^:=...;

Now, after file mapping isn't neede anymore, you should do following in DLL finalization:

UnmapViewOfFile(Base);
CloseHandle(hNewMap);

I haven't tested this yet, but it should work.

best regards
Benjamin Kalytta

Re:Memory mapped files in win32 free pascal


Quote
Peter H.M. Brooks wrote in message ...
>I have just recently started doing some work in win32 - my previous
>programming has been with Unix.

>I want to share memory between two processes and use free Pascal to do
>it.

>I first read some old documentation that led me to believe that DLLs
>would be the answer - they are required, but the code is copied into
>each process that rather defeats the object of a shared library.

The code is loaded just once, and the physical memory is
mapped into the logical memory space of processes as often
as needed.

(Don't worry. Contrary to what people may have told you,
Windows _is_ sane in some respects.)

Quote
>So, I now realise that I need to use memory mapped files,
>createfilemapping, openfilemapping and mapviewoffileex - so that I can
>map the file (actually the pagefile) to my variable space and have the
>inter-process data sharing that I need. The only example I can find is
>in c, on the ms site.

It's not hard to translate C to Object Pascal.

Anyway, you want to read around a bit in the win32 help
file supplied with Delphi. Start with GlobalAlloc and
read some topics near that. I don't think you have to
use memory-mapped files to get shared memory (although
it'll work).

Groetjes,
Maarten Wiltink

Re:Memory mapped files in win32 free pascal


Quote
"Benjamin Kalytta" <spam...@fight.at> wrote in message

news:aa0qvh$jdo$06$1@news.t-online.com...
Quote
> > Surely people are doing this in free Pascal all the time? Does
anybody
> > have some sample code that simply arranges for the global variables
in
> > the DLL to be available as shared memory to all processes using the
DLL?

> hFile=A valid file handle
> SizeToMap=Size of data that should be mapped

hMap:=CreateFileMapping(hFile,nil,PAGE_READWRITE,0,SizeToMap,'MyFileMapp
ingObject');
Quote

> Each DLL that should access this shared memory do following in
initialization:

> Base=pointer to suggested memory address where the data will be mapped
> into current address space.

hNewMap:=OpenFileMapping(FILE_MAP_ALL_ACCESS,false,'MyFileMappingObject'
);
Quote

MapViewOfFileEx(hNewMap,FILE_MAP_ALL_ACCESS,0,OffsetInFileToMap,NumberOf
BytesToMap,Base);
Quote

> Then add folowing variable export:

> exports
>   Base name 'MySharedMemory';

So is there a sensible way to map Base=base of DLL variable space, so
that all global variables are mapped in?

I suppose that:

type
base_t = ^base_t;

var
base : base_t;
..
base := ^base; then use 'base' as above, should do it.

Quote

> So, each process that loads this DLL can access this shared memory by
DLL export
> 'MySharedMemory'.
> There is no possibility to import variables in Delphi, there you
should use
> GetProcAddress(0,'MySharedMemory');
> In Free pascal it should be possible to declare a import variable:

> var
>   MySharedMemory:pointer;external 'My.Dll' name 'MySharedMemory';

> Accessing shared data is done by PMyType(MySharedMemory)^:=...;

Yes, though I thought that you could just do:

In DLL:

type
some_type = any type def. X.

var
fred : some_type;

export
fred;

In Other processes:

type
some_type = X as above;

var
fred : some_type external 'Mydll' name 'FRED';

and then just access fred as normal.

Quote

> Now, after file mapping isn't neede anymore, you should do following

in DLL finalization:
Quote

> UnmapViewOfFile(Base);
> CloseHandle(hNewMap);

> I haven't tested this yet, but it should work.

Thank you, yes, it looks as if it should. I will try it.

I would only need to unmap the memory when the processes are going to
die - which should only happen at shutdown, so I don't think I really
have to bother.

--
Great men are almost always bad men.
Lord Acton

Re:Memory mapped files in win32 free pascal


Quote
"Maarten Wiltink" <maar...@kittensandcats.net> wrote in message
> Peter H.M. Brooks wrote in message ...
> >I have just recently started doing some work in win32 - my previous
> >programming has been with Unix.

> >I want to share memory between two processes and use free Pascal to
do
> >it.

> >I first read some old documentation that led me to believe that DLLs
> >would be the answer - they are required, but the code is copied into
> >each process that rather defeats the object of a shared library.

> The code is loaded just once, and the physical memory is
> mapped into the logical memory space of processes as often
> as needed.

> (Don't worry. Contrary to what people may have told you,
> Windows _is_ sane in some respects.)

Yes, but it is different physical memory in each process that accesses
the DLL - unlike windows 3. So the variables cannot be dynamically
altered in one process and show up altered in another as truly global
variables should.

- Show quoted text -

Quote

> >So, I now realise that I need to use memory mapped files,
> >createfilemapping, openfilemapping and mapviewoffileex - so that I
can
> >map the file (actually the pagefile) to my variable space and have
the
> >inter-process data sharing that I need. The only example I can find
is
> >in c, on the ms site.

> It's not hard to translate C to Object Pascal.

> Anyway, you want to read around a bit in the win32 help
> file supplied with Delphi. Start with GlobalAlloc and
> read some topics near that. I don't think you have to
> use memory-mapped files to get shared memory (although
> it'll work).

Thank you - I'll have a look at that, but, according to the microsoft
developer web-site
http://msdn.microsoft.com/library/devault.asp?usl=/library/en-us/dngenli
b/html/msdn_manamemo.asp this is
the only method that exists to share memory dynamically between
processes is memory mapped files. At the bottom of this page it says 'A
memory-mapped file can also be mapped by more than one application
simultaneously. This  represents the only mechanism for two or more
processes to directly share data in Windows NT.'

It may well be wrong, but it is pretty direct - and the page is
copyright Microsoft 2002, so it is a fairly recent article.

As I said, though, I am amazed that there aren't a lot more examples of
this, and discussions of this on the Internet. If you have a daemon and
many client processes (common in server type applications) I would have
thought it was the most natural way of sharing status information,
locking things and keeping semi-static data (stuff that varies every
hour or so).

--
Great men are almost always bad men.
Lord Acton

Re:Memory mapped files in win32 free pascal


"Peter H.M. Brooks" <pe...@new.co.za> wrote in message
news:aa1luj$a1a$1@ctb-nnrp2.saix.net...

Quote

> Thank you - I'll have a look at that, but, according to the microsoft
> developer web-site
> http://msdn.microsoft.com/library/devault.asp?usl=/library/en-us/dngenli
> b/html/msdn_manamemo.asp this is
> the only method that exists to share memory dynamically between
> processes is memory mapped files. At the bottom of this page it says 'A
> memory-mapped file can also be mapped by more than one application
> simultaneously. This  represents the only mechanism for two or more
> processes to directly share data in Windows NT.'

> It may well be wrong, but it is pretty direct - and the page is
> copyright Microsoft 2002, so it is a fairly recent article.

> As I said, though, I am amazed that there aren't a lot more examples of
> this, and discussions of this on the Internet. If you have a daemon and
> many client processes (common in server type applications) I would have
> thought it was the most natural way of sharing status information,
> locking things and keeping semi-static data (stuff that varies every
> hour or so).

    Pipes are also useful for IPC.

--

-GJC
-gchan...@TheWorld.com

-War is the last resort of the incompetent.

Re:Memory mapped files in win32 free pascal


Quote
"Gary Chanson" <gchan...@no.spam.TheWorld.com> wrote in message

news:aa4dbj$lqm$1@pcls4.std.com...
Quote

> "Peter H.M. Brooks" <pe...@new.co.za> wrote in message
> news:aa1luj$a1a$1@ctb-nnrp2.saix.net...

> > Thank you - I'll have a look at that, but, according to the
microsoft
> > developer web-site

http://msdn.microsoft.com/library/devault.asp?usl=/library/en-us/dngenli
Quote
> > b/html/msdn_manamemo.asp this is
> > the only method that exists to share memory dynamically between
> > processes is memory mapped files. At the bottom of this page it says
'A
> > memory-mapped file can also be mapped by more than one application
> > simultaneously. This  represents the only mechanism for two or more
> > processes to directly share data in Windows NT.'

> > It may well be wrong, but it is pretty direct - and the page is
> > copyright Microsoft 2002, so it is a fairly recent article.

> > As I said, though, I am amazed that there aren't a lot more examples
of
> > this, and discussions of this on the Internet. If you have a daemon
and
> > many client processes (common in server type applications) I would
have
> > thought it was the most natural way of sharing status information,
> > locking things and keeping semi-static data (stuff that varies every
> > hour or so).

>     Pipes are also useful for IPC.

Yes, they certainly are - in Unix they are often used. They are
sequential, though, which makes them a lot less appealing for sharing
this sort of data.

I am though, amazed that so few people seem to have used the memory
mapped files that there is not a single example to be found in FP
anywhere!

--
Great men are almost always bad men.
Lord Acton

Re:Memory mapped files in win32 free pascal


"Peter H.M. Brooks" <pe...@new.co.za> wrote in message
news:aa1luj$a1a$1@ctb-nnrp2.saix.net...

Quote
> the only method that exists to share memory dynamically between
> processes is memory mapped files. At the bottom of this page it says 'A
> memory-mapped file can also be mapped by more than one application
> simultaneously. This  represents the only mechanism for two or more
> processes to directly share data in Windows NT.'

> It may well be wrong, but it is pretty direct - and the page is
> copyright Microsoft 2002, so it is a fairly recent article.

This is correct and the examples are out there but you should search
Win32API groups since this is not exactly a Delphi issue and it all comees
to converting C API calls to Pascal. I can send you an example if you mail
me, but then again, Benjamin gave you a  good answer.

Re:Memory mapped files in win32 free pascal


Quote
"Smola" <supersmola...@VEinet.hr> wrote in message

> "Peter H.M. Brooks" <pe...@new.co.za> wrote in message
> news:aa1luj$a1a$1@ctb-nnrp2.saix.net...

> > the only method that exists to share memory dynamically between
> > processes is memory mapped files. At the bottom of this page it says
'A
> > memory-mapped file can also be mapped by more than one application
> > simultaneously. This  represents the only mechanism for two or more
> > processes to directly share data in Windows NT.'

> > It may well be wrong, but it is pretty direct - and the page is
> > copyright Microsoft 2002, so it is a fairly recent article.

> This is correct and the examples are out there but you should search
> Win32API groups since this is not exactly a Delphi issue and it all
comees
> to converting C API calls to Pascal. I can send you an example if you
mail
> me, but then again, Benjamin gave you a  good answer.

Yes, thank you for that. I agree that that is how it is at the moment.

I wonder if it would be a useful extension to the language to be able to
declare an export as 'shared' and have the compiler set up the mapping
so that all processes using the same DLL would have access to the
exported variables as shared global space. If it was an option I would
imagine a lot more people would use it - this memory mapped file stuff
does seem a long way round to do something quite simple!

--
Great men are almost always bad men.
Lord Acton

Re:Memory mapped files in win32 free pascal


Peter H.M. Brooks <pe...@new.co.za> wrote in message
news:aa630h$gh0$1@ctb-nnrp2.saix.net...

Quote

> "Smola" <supersmola...@VEinet.hr> wrote in message

> > "Peter H.M. Brooks" <pe...@new.co.za> wrote in message
> > news:aa1luj$a1a$1@ctb-nnrp2.saix.net...

> > > the only method that exists to share memory dynamically between
> > > processes is memory mapped files. At the bottom of this page it says
> 'A
> > > memory-mapped file can also be mapped by more than one application
> > > simultaneously. This  represents the only mechanism for two or more
> > > processes to directly share data in Windows NT.'

> > > It may well be wrong, but it is pretty direct - and the page is
> > > copyright Microsoft 2002, so it is a fairly recent article.

> > This is correct and the examples are out there but you should search
> > Win32API groups since this is not exactly a Delphi issue and it all
> comees
> > to converting C API calls to Pascal. I can send you an example if you
> mail
> > me, but then again, Benjamin gave you a  good answer.

> Yes, thank you for that. I agree that that is how it is at the moment.

> I wonder if it would be a useful extension to the language to be able to
> declare an export as 'shared' and have the compiler set up the mapping
> so that all processes using the same DLL would have access to the
> exported variables as shared global space. If it was an option I would
> imagine a lot more people would use it - this memory mapped file stuff
> does seem a long way round to do something quite simple!

I've been waiting for someone else to wade in here, but, I guess its only
myself  that's intrigued by this one! ISTM that there are very good reasons
to deny the trivial mapping of one process's memory space into another. I
don't know the inner workings of the modern PC at all, but have designed
extensively with the 68K family in which, at the hardware level, process
memory spaces are rigorously mapped (if you choose to do it that way) and
there is no trivial way to get from one space to another. I have always
assumed that the use of file mapped memory in W32 is a kludge to get around
the problem, since IO memory is the only memory space separate processes can
hold in common. Perhaps you can tell me otherwise........ perhaps this is
off topic for this ng :-)

However, lets leave the nuts and bolts level. I have never found the need
for file mapped memory, and judging from the responses to this thread, it
can't be something people do very often. It's true I have considered it, but
there has always been a tidier solution. For example, I often need two
processes to communicate in control/dataflow situations that occur in
industrial instrument control. Here I've found COM to provide very simple
solutions from the point of view of my time and effort.  This being
especially so when I have to provide a programming control interface for a
third party, as the interface publishes useful things like eneumerated
constants and data types.

You mention a possible scenaraio where you have a large number of processes
being generated in response to incoming data connection requests, and these
processes need to be able to share information. Isn't this why the OS allows
you to spawn child processes within your main process space? Certainly I
have found this a very useable approach. I won't say easy, as the
synchronisation issues demand careful thought.

My two centums .... no more!

Dave

Re:Memory mapped files in win32 free pascal


Quote
"David Reeve" <drscienti...@powerup.com.au> wrote in message
> > I wonder if it would be a useful extension to the language to be
able to
> > declare an export as 'shared' and have the compiler set up the
mapping
> > so that all processes using the same DLL would have access to the
> > exported variables as shared global space. If it was an option I
would
> > imagine a lot more people would use it - this memory mapped file
stuff
> > does seem a long way round to do something quite simple!

> I've been waiting for someone else to wade in here, but, I guess its
only
> myself  that's intrigued by this one! ISTM that there are very good
reasons
> to deny the trivial mapping of one process's memory space into
another. I
> don't know the inner workings of the modern PC at all, but have
designed
> extensively with the 68K family in which, at the hardware level,
process
> memory spaces are rigorously mapped (if you choose to do it that way)
and
> there is no trivial way to get from one space to another. I have
always
> assumed that the use of file mapped memory in W32 is a kludge to get
around
> the problem, since IO memory is the only memory space separate
processes can
> hold in common. Perhaps you can tell me otherwise........ perhaps this
is
> off topic for this ng :-)

There are very good security reasons for programs normally to be mapped
only to their own and system space, for I/O and such like. That is why
the systems are set up like that.

There are also good reasons for wishing to have processes able to share
memory (all OSs allow some form of shared memory, Unix has a quite
sophisticated memory sharing system).

Most of the situations I have seen where this is useful involve
asynchronos requests from different sources, so these cannot be handled
by one process tree - unless you have intermediate processes and
communicate through some other messier method.

- Show quoted text -

Quote

> However, lets leave the nuts and bolts level. I have never found the
need
> for file mapped memory, and judging from the responses to this thread,
it
> can't be something people do very often. It's true I have considered
it, but
> there has always been a tidier solution. For example, I often need two
> processes to communicate in control/dataflow situations that occur in
> industrial instrument control. Here I've found COM to provide very
simple
> solutions from the point of view of my time and effort.  This being
> especially so when I have to provide a programming control interface
for a
> third party, as the interface publishes useful things like eneumerated
> constants and data types.

Yes, I agree, the COM interface is starting to provide a reasonable
IPC - and that is important as well.
Quote

> You mention a possible scenaraio where you have a large number of
processes
> being generated in response to incoming data connection requests, and
these
> processes need to be able to share information. Isn't this why the OS
allows
> you to spawn child processes within your main process space? Certainly
I
> have found this a very useable approach. I won't say easy, as the
> synchronisation issues demand careful thought.

Yes, this does work well when you use the socket/port model - if you
can!

--
Great men are almost always bad men.
Lord Acton

Other Threads