Board index » delphi » HUGE MEMORY LEAK in Interbase makes product completely unusable

HUGE MEMORY LEAK in Interbase makes product completely unusable

Over the weekend, I had the pleasure to find the location of an enormous
memory leak in Interbase. I searched the newsgroups, and other threads
related to IB, and found several people complaining about such leaks.
However, all of the responses from Borland folks have stated that the issue
was due to unclosed connections, etc.

Well, I have a reproducible test case that is only a couple lines.

My configuration is as follows:

Interbase 6.01 (Open Source)
Interbase Client 1.60 (I have also reproduced with 2.0) (Open Source)
JDBC Drivers 1.60
Windows 2000

The problem appears to show itself when using JDBC Statement operations that
perform updates (delete, update, etc.). Here is the sample code that will
allow you to deplete your computer of all its memory.

try
{
    Connection con = /* Create JDBC connection here using
"interbase.interclient.Driver" */
    for( int i = 0; i < 1000; i++ )
    {
        int newVal = (int)(Math.random() * 100000);
        Statement st = con.createStatement();
        st.executeUpdate( "update foo set foocol = " + newVal + " where
foocol > 0" );
        st.close();
        con.commit();
    }
    con.close();

Quote
}

catch( Exception e )
{
    e.printStackTrace();

Quote
}

The DDL is the following:    create table foo( foocol integer );

I insert a row before testing, but it doesn't matter, the leak occurs in
either case.

When you run this program, watch the memory consumption of ibserver.exe and
interserver.exe (the connection). Both of these processes will leak memory
at a huge rate. When the program exits, none of the memory is reclaimed.

The most interesting part is that if you change the Statement above to a
PreparedStatement, the processes hardly leak at all. If they do leak it is
at a very low rate.

If there is anything I am missing, or something I am doing wrong, please let
me know.

--Chris.

 

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


What memory proof program are using you ? Task Manager of Win2K isn't
suitable for that task. I don't know exactly how Win manages memory but
according to previous posts memory released by preocesses isn't claimed
immediatle by OS. And it make sense since getting OS memory (on any
platform) is a very resource intensive task, then I guess that Win OS
doesn't return immediate the memory becuase is very probably that the
process will ask for memory again in the near future.

--
Sergio Samayoa
Lgica Software
http://www.geocities.com/logicasw/

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Interesting... However, the fact that the machine is crawling due to low
memory and swapping after a 12 hour period makes me think that the memory is
never returned to the OS.

--Chris.

Quote
"Sergio Samayoa" <serg...@terra.com.gt> wrote in message

news:3b378a0e_1@dnews...
Quote
> What memory proof program are using you ? Task Manager of Win2K isn't
> suitable for that task. I don't know exactly how Win manages memory but
> according to previous posts memory released by preocesses isn't claimed
> immediatle by OS. And it make sense since getting OS memory (on any
> platform) is a very resource intensive task, then I guess that Win OS
> doesn't return immediate the memory becuase is very probably that the
> process will ask for memory again in the near future.

> --
> Sergio Samayoa
> Lgica Software
> http://www.geocities.com/logicasw/

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


My understanding here is that releasing memory back to the OS is a function
of the implementation of the malloc library (or whatever memory scheme the
program is using).  Unix does the same, malloc calls sbrk to gram system
memory, but must return it later.  Calling free doesn't mean that the OS has
regained any memory, it just means that the pool or free memory that malloc
is using has grown.

Back to Chris' original point, if this was a general memory freeing problem
then I would have expected the problem to a) have been more obvious and b)
to have applied to more than that just update operations.

Guy

Quote
"Sergio Samayoa" <serg...@terra.com.gt> wrote in message

news:3b378a0e_1@dnews...Guy
Quote
> What memory proof program are using you ? Task Manager of Win2K isn't
> suitable for that task. I don't know exactly how Win manages memory but
> according to previous posts memory released by preocesses isn't claimed
> immediatle by OS. And it make sense since getting OS memory (on any
> platform) is a very resource intensive task, then I guess that Win OS
> doesn't return immediate the memory becuase is very probably that the
> process will ask for memory again in the near future.

> --
> Sergio Samayoa
> Lgica Software
> http://www.geocities.com/logicasw/

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


I forget about IB's config. IB uses as much memory as you configure in DB
pages. May be you have too high this setting makign the Win to swap to disk.
All my GDB's uses 4 Kb pages and I have setting to cache just 2048 pages,
then one opened GDB uses as much as 8 Mb of RAM. Even if I do a huge insert
or update in that GDB file, IB allways use 8 Mb or less. Check how many
opened GDB files you have, how many memory are used by another services and
how much cache is used by each opened GDB.

--
Sergio Samayoa
Lgica Software
http://www.geocities.com/logicasw/

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Well, if you look at the code that I have with the original post, the
transaction is only one update statement. Also, the table can be completely
empty, and the memory is still leaked. Thus, I don't think it has anything
to do with data page size, etc. Also, memory is leaked from the connect
(interserver.exe), which probably wouldn't be related to the page size.

We are using a single GDB file. In this test case, you only have to have the
"foo" table, which can be completely empty.

--Chris.

Quote
"Sergio Samayoa" <serg...@terra.com.gt> wrote in message

news:3b3793ca$1_1@dnews...
Quote
> I forget about IB's config. IB uses as much memory as you configure in DB
> pages. May be you have too high this setting makign the Win to swap to
disk.
> All my GDB's uses 4 Kb pages and I have setting to cache just 2048 pages,
> then one opened GDB uses as much as 8 Mb of RAM. Even if I do a huge
insert
> or update in that GDB file, IB allways use 8 Mb or less. Check how many
> opened GDB files you have, how many memory are used by another services
and
> how much cache is used by each opened GDB.

> --
> Sergio Samayoa
> Lgica Software
> http://www.geocities.com/logicasw/

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Actually both processes are showing the leak (ibserver.exe and
interserver.exe). As far as not unpreparing goes, the only method available
is close() to deallocate. Here is the Javadoc:

 /**
  * Releases this <code>Statement</code> object's database
  * and JDBC resources immediately instead of waiting for
  * this to happen when it is automatically closed.
  * It is generally good practice to release resources as soon as
  * you are finished with them to avoid tying up database
  * resources.
   * <P><B>Note:</B> A <code>Statement</code> object is automatically closed
when it is
  * garbage collected. When a <code>Statement</code> object is closed, its
current
   * <code>ResultSet</code> object, if one exists, is also closed.
   *
   * @exception SQLException if a database access error occurs
   */

Also, note that it does not leak memory when I replace "Statement" with a
"PreparedStatement". Therefore, I am guessing that my code is correct.

--Chris.

Quote
"Jason Wharton" <jwhar...@ibobjects.com> wrote in message

news:3b37991b$1_2@dnews...
Quote
> Which process is showing the memory growth?

> IBServer or the server for Interserver that hosts Interclient?

> You are creating and allocating a statement and merely closing it (which
> relates to a cursor only). How can it actually unprepare and deallocate
the
> statement? Apparently JDBC expects you to explicitly handle this detail.

> I'm a Delphi guy so I could be wrong... but I am very confident your code
is
> wrong.

> Regards,
> Jason Wharton
> CPS - Mesa AZ
> http://www.ibobjects.com

> "Chris Smith" <csm...@tripwire.com> wrote in message
> news:3b37834f$1_2@dnews...
> > Over the weekend, I had the pleasure to find the location of an enormous
> > memory leak in Interbase. I searched the newsgroups, and other threads
> > related to IB, and found several people complaining about such leaks.
> > However, all of the responses from Borland folks have stated that the
> issue
> > was due to unclosed connections, etc.

> > Well, I have a reproducible test case that is only a couple lines.

> > My configuration is as follows:

> > Interbase 6.01 (Open Source)
> > Interbase Client 1.60 (I have also reproduced with 2.0) (Open Source)
> > JDBC Drivers 1.60
> > Windows 2000

> > The problem appears to show itself when using JDBC Statement operations
> that
> > perform updates (delete, update, etc.). Here is the sample code that
will
> > allow you to deplete your computer of all its memory.

> > try
> > {
> >     Connection con = /* Create JDBC connection here using
> > "interbase.interclient.Driver" */
> >     for( int i = 0; i < 1000; i++ )
> >     {
> >         int newVal = (int)(Math.random() * 100000);
> >         Statement st = con.createStatement();
> >         st.executeUpdate( "update foo set foocol = " + newVal + " where
> > foocol > 0" );
> >         st.close();
> >         con.commit();
> >     }
> >     con.close();
> > }
> > catch( Exception e )
> > {
> >     e.printStackTrace();
> > }

> > The DDL is the following:    create table foo( foocol integer );

> > I insert a row before testing, but it doesn't matter, the leak occurs in
> > either case.

> > When you run this program, watch the memory consumption of ibserver.exe
> and
> > interserver.exe (the connection). Both of these processes will leak
memory
> > at a huge rate. When the program exits, none of the memory is reclaimed.

> > The most interesting part is that if you change the Statement above to a
> > PreparedStatement, the processes hardly leak at all. If they do leak it
is
> > at a very low rate.

> > If there is anything I am missing, or something I am doing wrong, please
> let
> > me know.

> > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Which process is showing the memory growth?

IBServer or the server for Interserver that hosts Interclient?

You are creating and allocating a statement and merely closing it (which
relates to a cursor only). How can it actually unprepare and deallocate the
statement? Apparently JDBC expects you to explicitly handle this detail.

I'm a Delphi guy so I could be wrong... but I am very confident your code is
wrong.

Regards,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com

Quote
"Chris Smith" <csm...@tripwire.com> wrote in message

news:3b37834f$1_2@dnews...
Quote
> Over the weekend, I had the pleasure to find the location of an enormous
> memory leak in Interbase. I searched the newsgroups, and other threads
> related to IB, and found several people complaining about such leaks.
> However, all of the responses from Borland folks have stated that the
issue
> was due to unclosed connections, etc.

> Well, I have a reproducible test case that is only a couple lines.

> My configuration is as follows:

> Interbase 6.01 (Open Source)
> Interbase Client 1.60 (I have also reproduced with 2.0) (Open Source)
> JDBC Drivers 1.60
> Windows 2000

> The problem appears to show itself when using JDBC Statement operations
that
> perform updates (delete, update, etc.). Here is the sample code that will
> allow you to deplete your computer of all its memory.

> try
> {
>     Connection con = /* Create JDBC connection here using
> "interbase.interclient.Driver" */
>     for( int i = 0; i < 1000; i++ )
>     {
>         int newVal = (int)(Math.random() * 100000);
>         Statement st = con.createStatement();
>         st.executeUpdate( "update foo set foocol = " + newVal + " where
> foocol > 0" );
>         st.close();
>         con.commit();
>     }
>     con.close();
> }
> catch( Exception e )
> {
>     e.printStackTrace();
> }

> The DDL is the following:    create table foo( foocol integer );

> I insert a row before testing, but it doesn't matter, the leak occurs in
> either case.

> When you run this program, watch the memory consumption of ibserver.exe
and
> interserver.exe (the connection). Both of these processes will leak memory
> at a huge rate. When the program exits, none of the memory is reclaimed.

> The most interesting part is that if you change the Statement above to a
> PreparedStatement, the processes hardly leak at all. If they do leak it is
> at a very low rate.

> If there is anything I am missing, or something I am doing wrong, please
let
> me know.

> --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


I would suspect this as a bug in the interserver/interclient implementation
then.
What you are doing is chewing up statement handles and for some reason they
are not being deallocated.
Is there any way you can know if garbage collection is taking place.
I sure would hate to think that database resources are being tied to JAVA
garbage collection.
There should be a way that you can explicitly demand database resources to
be released before an object is put on the garbage collection queue.

Sure glad I use Delphi (and not IBX).

FWIW,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com

Quote
"Chris Smith" <csm...@tripwire.com> wrote in message

news:3b379a7d$1_2@dnews...
Quote
> Actually both processes are showing the leak (ibserver.exe and
> interserver.exe). As far as not unpreparing goes, the only method
available
> is close() to deallocate. Here is the Javadoc:

>  /**
>   * Releases this <code>Statement</code> object's database
>   * and JDBC resources immediately instead of waiting for
>   * this to happen when it is automatically closed.
>   * It is generally good practice to release resources as soon as
>   * you are finished with them to avoid tying up database
>   * resources.
>    * <P><B>Note:</B> A <code>Statement</code> object is automatically
closed
> when it is
>   * garbage collected. When a <code>Statement</code> object is closed, its
> current
>    * <code>ResultSet</code> object, if one exists, is also closed.
>    *
>    * @exception SQLException if a database access error occurs
>    */

> Also, note that it does not leak memory when I replace "Statement" with a
> "PreparedStatement". Therefore, I am guessing that my code is correct.

> --Chris.

> "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> news:3b37991b$1_2@dnews...
> > Which process is showing the memory growth?

> > IBServer or the server for Interserver that hosts Interclient?

> > You are creating and allocating a statement and merely closing it (which
> > relates to a cursor only). How can it actually unprepare and deallocate
> the
> > statement? Apparently JDBC expects you to explicitly handle this detail.

> > I'm a Delphi guy so I could be wrong... but I am very confident your
code
> is
> > wrong.

> > Regards,
> > Jason Wharton
> > CPS - Mesa AZ
> > http://www.ibobjects.com

> > "Chris Smith" <csm...@tripwire.com> wrote in message
> > news:3b37834f$1_2@dnews...
> > > Over the weekend, I had the pleasure to find the location of an
enormous
> > > memory leak in Interbase. I searched the newsgroups, and other threads
> > > related to IB, and found several people complaining about such leaks.
> > > However, all of the responses from Borland folks have stated that the
> > issue
> > > was due to unclosed connections, etc.

> > > Well, I have a reproducible test case that is only a couple lines.

> > > My configuration is as follows:

> > > Interbase 6.01 (Open Source)
> > > Interbase Client 1.60 (I have also reproduced with 2.0) (Open Source)
> > > JDBC Drivers 1.60
> > > Windows 2000

> > > The problem appears to show itself when using JDBC Statement
operations
> > that
> > > perform updates (delete, update, etc.). Here is the sample code that
> will
> > > allow you to deplete your computer of all its memory.

> > > try
> > > {
> > >     Connection con = /* Create JDBC connection here using
> > > "interbase.interclient.Driver" */
> > >     for( int i = 0; i < 1000; i++ )
> > >     {
> > >         int newVal = (int)(Math.random() * 100000);
> > >         Statement st = con.createStatement();
> > >         st.executeUpdate( "update foo set foocol = " + newVal + "
where
> > > foocol > 0" );
> > >         st.close();
> > >         con.commit();
> > >     }
> > >     con.close();
> > > }
> > > catch( Exception e )
> > > {
> > >     e.printStackTrace();
> > > }

> > > The DDL is the following:    create table foo( foocol integer );

> > > I insert a row before testing, but it doesn't matter, the leak occurs
in
> > > either case.

> > > When you run this program, watch the memory consumption of
ibserver.exe
> > and
> > > interserver.exe (the connection). Both of these processes will leak
> memory
> > > at a huge rate. When the program exits, none of the memory is
reclaimed.

> > > The most interesting part is that if you change the Statement above to
a
> > > PreparedStatement, the processes hardly leak at all. If they do leak
it
> is
> > > at a very low rate.

> > > If there is anything I am missing, or something I am doing wrong,
please
> > let
> > > me know.

> > > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


I don't think it is tied at all to Java garbage collection. When the test
program exits, all of the memory is still consumed by the IB processes.

--Chris.

Quote
"Jason Wharton" <jwhar...@ibobjects.com> wrote in message

news:3b379e24$1_2@dnews...
Quote
> I would suspect this as a bug in the interserver/interclient
implementation
> then.
> What you are doing is chewing up statement handles and for some reason
they
> are not being deallocated.
> Is there any way you can know if garbage collection is taking place.
> I sure would hate to think that database resources are being tied to JAVA
> garbage collection.
> There should be a way that you can explicitly demand database resources to
> be released before an object is put on the garbage collection queue.

> Sure glad I use Delphi (and not IBX).

> FWIW,
> Jason Wharton
> CPS - Mesa AZ
> http://www.ibobjects.com

> "Chris Smith" <csm...@tripwire.com> wrote in message
> news:3b379a7d$1_2@dnews...
> > Actually both processes are showing the leak (ibserver.exe and
> > interserver.exe). As far as not unpreparing goes, the only method
> available
> > is close() to deallocate. Here is the Javadoc:

> >  /**
> >   * Releases this <code>Statement</code> object's database
> >   * and JDBC resources immediately instead of waiting for
> >   * this to happen when it is automatically closed.
> >   * It is generally good practice to release resources as soon as
> >   * you are finished with them to avoid tying up database
> >   * resources.
> >    * <P><B>Note:</B> A <code>Statement</code> object is automatically
> closed
> > when it is
> >   * garbage collected. When a <code>Statement</code> object is closed,
its
> > current
> >    * <code>ResultSet</code> object, if one exists, is also closed.
> >    *
> >    * @exception SQLException if a database access error occurs
> >    */

> > Also, note that it does not leak memory when I replace "Statement" with
a
> > "PreparedStatement". Therefore, I am guessing that my code is correct.

> > --Chris.

> > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > news:3b37991b$1_2@dnews...
> > > Which process is showing the memory growth?

> > > IBServer or the server for Interserver that hosts Interclient?

> > > You are creating and allocating a statement and merely closing it
(which
> > > relates to a cursor only). How can it actually unprepare and
deallocate
> > the
> > > statement? Apparently JDBC expects you to explicitly handle this
detail.

> > > I'm a Delphi guy so I could be wrong... but I am very confident your
> code
> > is
> > > wrong.

> > > Regards,
> > > Jason Wharton
> > > CPS - Mesa AZ
> > > http://www.ibobjects.com

> > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > news:3b37834f$1_2@dnews...
> > > > Over the weekend, I had the pleasure to find the location of an
> enormous
> > > > memory leak in Interbase. I searched the newsgroups, and other
threads
> > > > related to IB, and found several people complaining about such
leaks.
> > > > However, all of the responses from Borland folks have stated that
the
> > > issue
> > > > was due to unclosed connections, etc.

> > > > Well, I have a reproducible test case that is only a couple lines.

> > > > My configuration is as follows:

> > > > Interbase 6.01 (Open Source)
> > > > Interbase Client 1.60 (I have also reproduced with 2.0) (Open
Source)
> > > > JDBC Drivers 1.60
> > > > Windows 2000

> > > > The problem appears to show itself when using JDBC Statement
> operations
> > > that
> > > > perform updates (delete, update, etc.). Here is the sample code that
> > will
> > > > allow you to deplete your computer of all its memory.

> > > > try
> > > > {
> > > >     Connection con = /* Create JDBC connection here using
> > > > "interbase.interclient.Driver" */
> > > >     for( int i = 0; i < 1000; i++ )
> > > >     {
> > > >         int newVal = (int)(Math.random() * 100000);
> > > >         Statement st = con.createStatement();
> > > >         st.executeUpdate( "update foo set foocol = " + newVal + "
> where
> > > > foocol > 0" );
> > > >         st.close();
> > > >         con.commit();
> > > >     }
> > > >     con.close();
> > > > }
> > > > catch( Exception e )
> > > > {
> > > >     e.printStackTrace();
> > > > }

> > > > The DDL is the following:    create table foo( foocol integer );

> > > > I insert a row before testing, but it doesn't matter, the leak
occurs
> in
> > > > either case.

> > > > When you run this program, watch the memory consumption of
> ibserver.exe
> > > and
> > > > interserver.exe (the connection). Both of these processes will leak
> > memory
> > > > at a huge rate. When the program exits, none of the memory is
> reclaimed.

> > > > The most interesting part is that if you change the Statement above
to
> a
> > > > PreparedStatement, the processes hardly leak at all. If they do leak
> it
> > is
> > > > at a very low rate.

> > > > If there is anything I am missing, or something I am doing wrong,
> please
> > > let
> > > > me know.

> > > > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Then it is for sure a bug in the interserver/interclient driver. Stop and
start the interserver process and I bet within seconds the ibserver process
recovers handsomely.

--
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com

Quote
"Chris Smith" <csm...@tripwire.com> wrote in message

news:3b37a005$1_1@dnews...
Quote
> I don't think it is tied at all to Java garbage collection. When the test
> program exits, all of the memory is still consumed by the IB processes.

> --Chris.

> "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> news:3b379e24$1_2@dnews...
> > I would suspect this as a bug in the interserver/interclient
> implementation
> > then.
> > What you are doing is chewing up statement handles and for some reason
> they
> > are not being deallocated.
> > Is there any way you can know if garbage collection is taking place.
> > I sure would hate to think that database resources are being tied to
JAVA
> > garbage collection.
> > There should be a way that you can explicitly demand database resources
to
> > be released before an object is put on the garbage collection queue.

> > Sure glad I use Delphi (and not IBX).

> > FWIW,
> > Jason Wharton
> > CPS - Mesa AZ
> > http://www.ibobjects.com

> > "Chris Smith" <csm...@tripwire.com> wrote in message
> > news:3b379a7d$1_2@dnews...
> > > Actually both processes are showing the leak (ibserver.exe and
> > > interserver.exe). As far as not unpreparing goes, the only method
> > available
> > > is close() to deallocate. Here is the Javadoc:

> > >  /**
> > >   * Releases this <code>Statement</code> object's database
> > >   * and JDBC resources immediately instead of waiting for
> > >   * this to happen when it is automatically closed.
> > >   * It is generally good practice to release resources as soon as
> > >   * you are finished with them to avoid tying up database
> > >   * resources.
> > >    * <P><B>Note:</B> A <code>Statement</code> object is automatically
> > closed
> > > when it is
> > >   * garbage collected. When a <code>Statement</code> object is closed,
> its
> > > current
> > >    * <code>ResultSet</code> object, if one exists, is also closed.
> > >    *
> > >    * @exception SQLException if a database access error occurs
> > >    */

> > > Also, note that it does not leak memory when I replace "Statement"
with
> a
> > > "PreparedStatement". Therefore, I am guessing that my code is correct.

> > > --Chris.

> > > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > > news:3b37991b$1_2@dnews...
> > > > Which process is showing the memory growth?

> > > > IBServer or the server for Interserver that hosts Interclient?

> > > > You are creating and allocating a statement and merely closing it
> (which
> > > > relates to a cursor only). How can it actually unprepare and
> deallocate
> > > the
> > > > statement? Apparently JDBC expects you to explicitly handle this
> detail.

> > > > I'm a Delphi guy so I could be wrong... but I am very confident your
> > code
> > > is
> > > > wrong.

> > > > Regards,
> > > > Jason Wharton
> > > > CPS - Mesa AZ
> > > > http://www.ibobjects.com

> > > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > > news:3b37834f$1_2@dnews...
> > > > > Over the weekend, I had the pleasure to find the location of an
> > enormous
> > > > > memory leak in Interbase. I searched the newsgroups, and other
> threads
> > > > > related to IB, and found several people complaining about such
> leaks.
> > > > > However, all of the responses from Borland folks have stated that
> the
> > > > issue
> > > > > was due to unclosed connections, etc.

> > > > > Well, I have a reproducible test case that is only a couple lines.

> > > > > My configuration is as follows:

> > > > > Interbase 6.01 (Open Source)
> > > > > Interbase Client 1.60 (I have also reproduced with 2.0) (Open
> Source)
> > > > > JDBC Drivers 1.60
> > > > > Windows 2000

> > > > > The problem appears to show itself when using JDBC Statement
> > operations
> > > > that
> > > > > perform updates (delete, update, etc.). Here is the sample code
that
> > > will
> > > > > allow you to deplete your computer of all its memory.

> > > > > try
> > > > > {
> > > > >     Connection con = /* Create JDBC connection here using
> > > > > "interbase.interclient.Driver" */
> > > > >     for( int i = 0; i < 1000; i++ )
> > > > >     {
> > > > >         int newVal = (int)(Math.random() * 100000);
> > > > >         Statement st = con.createStatement();
> > > > >         st.executeUpdate( "update foo set foocol = " + newVal + "
> > where
> > > > > foocol > 0" );
> > > > >         st.close();
> > > > >         con.commit();
> > > > >     }
> > > > >     con.close();
> > > > > }
> > > > > catch( Exception e )
> > > > > {
> > > > >     e.printStackTrace();
> > > > > }

> > > > > The DDL is the following:    create table foo( foocol integer );

> > > > > I insert a row before testing, but it doesn't matter, the leak
> occurs
> > in
> > > > > either case.

> > > > > When you run this program, watch the memory consumption of
> > ibserver.exe
> > > > and
> > > > > interserver.exe (the connection). Both of these processes will
leak
> > > memory
> > > > > at a huge rate. When the program exits, none of the memory is
> > reclaimed.

> > > > > The most interesting part is that if you change the Statement
above
> to
> > a
> > > > > PreparedStatement, the processes hardly leak at all. If they do
leak
> > it
> > > is
> > > > > at a very low rate.

> > > > > If there is anything I am missing, or something I am doing wrong,
> > please
> > > > let
> > > > > me know.

> > > > > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Actually, I tried that. Unfortunately, the ibserver.exe process does not
recover the memory.

--Chris.

Quote
"Jason Wharton" <jwhar...@ibobjects.com> wrote in message

news:3b37a3d0$1_1@dnews...
Quote
> Then it is for sure a bug in the interserver/interclient driver. Stop and
> start the interserver process and I bet within seconds the ibserver
process
> recovers handsomely.

> --
> Jason Wharton
> CPS - Mesa AZ
> http://www.ibobjects.com

> "Chris Smith" <csm...@tripwire.com> wrote in message
> news:3b37a005$1_1@dnews...
> > I don't think it is tied at all to Java garbage collection. When the
test
> > program exits, all of the memory is still consumed by the IB processes.

> > --Chris.

> > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > news:3b379e24$1_2@dnews...
> > > I would suspect this as a bug in the interserver/interclient
> > implementation
> > > then.
> > > What you are doing is chewing up statement handles and for some reason
> > they
> > > are not being deallocated.
> > > Is there any way you can know if garbage collection is taking place.
> > > I sure would hate to think that database resources are being tied to
> JAVA
> > > garbage collection.
> > > There should be a way that you can explicitly demand database
resources
> to
> > > be released before an object is put on the garbage collection queue.

> > > Sure glad I use Delphi (and not IBX).

> > > FWIW,
> > > Jason Wharton
> > > CPS - Mesa AZ
> > > http://www.ibobjects.com

> > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > news:3b379a7d$1_2@dnews...
> > > > Actually both processes are showing the leak (ibserver.exe and
> > > > interserver.exe). As far as not unpreparing goes, the only method
> > > available
> > > > is close() to deallocate. Here is the Javadoc:

> > > >  /**
> > > >   * Releases this <code>Statement</code> object's database
> > > >   * and JDBC resources immediately instead of waiting for
> > > >   * this to happen when it is automatically closed.
> > > >   * It is generally good practice to release resources as soon as
> > > >   * you are finished with them to avoid tying up database
> > > >   * resources.
> > > >    * <P><B>Note:</B> A <code>Statement</code> object is
automatically
> > > closed
> > > > when it is
> > > >   * garbage collected. When a <code>Statement</code> object is
closed,
> > its
> > > > current
> > > >    * <code>ResultSet</code> object, if one exists, is also closed.
> > > >    *
> > > >    * @exception SQLException if a database access error occurs
> > > >    */

> > > > Also, note that it does not leak memory when I replace "Statement"
> with
> > a
> > > > "PreparedStatement". Therefore, I am guessing that my code is
correct.

> > > > --Chris.

> > > > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > > > news:3b37991b$1_2@dnews...
> > > > > Which process is showing the memory growth?

> > > > > IBServer or the server for Interserver that hosts Interclient?

> > > > > You are creating and allocating a statement and merely closing it
> > (which
> > > > > relates to a cursor only). How can it actually unprepare and
> > deallocate
> > > > the
> > > > > statement? Apparently JDBC expects you to explicitly handle this
> > detail.

> > > > > I'm a Delphi guy so I could be wrong... but I am very confident
your
> > > code
> > > > is
> > > > > wrong.

> > > > > Regards,
> > > > > Jason Wharton
> > > > > CPS - Mesa AZ
> > > > > http://www.ibobjects.com

> > > > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > > > news:3b37834f$1_2@dnews...
> > > > > > Over the weekend, I had the pleasure to find the location of an
> > > enormous
> > > > > > memory leak in Interbase. I searched the newsgroups, and other
> > threads
> > > > > > related to IB, and found several people complaining about such
> > leaks.
> > > > > > However, all of the responses from Borland folks have stated
that
> > the
> > > > > issue
> > > > > > was due to unclosed connections, etc.

> > > > > > Well, I have a reproducible test case that is only a couple
lines.

> > > > > > My configuration is as follows:

> > > > > > Interbase 6.01 (Open Source)
> > > > > > Interbase Client 1.60 (I have also reproduced with 2.0) (Open
> > Source)
> > > > > > JDBC Drivers 1.60
> > > > > > Windows 2000

> > > > > > The problem appears to show itself when using JDBC Statement
> > > operations
> > > > > that
> > > > > > perform updates (delete, update, etc.). Here is the sample code
> that
> > > > will
> > > > > > allow you to deplete your computer of all its memory.

> > > > > > try
> > > > > > {
> > > > > >     Connection con = /* Create JDBC connection here using
> > > > > > "interbase.interclient.Driver" */
> > > > > >     for( int i = 0; i < 1000; i++ )
> > > > > >     {
> > > > > >         int newVal = (int)(Math.random() * 100000);
> > > > > >         Statement st = con.createStatement();
> > > > > >         st.executeUpdate( "update foo set foocol = " + newVal +
"
> > > where
> > > > > > foocol > 0" );
> > > > > >         st.close();
> > > > > >         con.commit();
> > > > > >     }
> > > > > >     con.close();
> > > > > > }
> > > > > > catch( Exception e )
> > > > > > {
> > > > > >     e.printStackTrace();
> > > > > > }

> > > > > > The DDL is the following:    create table foo( foocol integer );

> > > > > > I insert a row before testing, but it doesn't matter, the leak
> > occurs
> > > in
> > > > > > either case.

> > > > > > When you run this program, watch the memory consumption of
> > > ibserver.exe
> > > > > and
> > > > > > interserver.exe (the connection). Both of these processes will
> leak
> > > > memory
> > > > > > at a huge rate. When the program exits, none of the memory is
> > > reclaimed.

> > > > > > The most interesting part is that if you change the Statement
> above
> > to
> > > a
> > > > > > PreparedStatement, the processes hardly leak at all. If they do
> leak
> > > it
> > > > is
> > > > > > at a very low rate.

> > > > > > If there is anything I am missing, or something I am doing
wrong,
> > > please
> > > > > let
> > > > > > me know.

> > > > > > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


        First, please do not multi-post your posts.  This violates Borland's
rules for the use of their newsgroups, which you can read here:

http://www.borland.com/newsgroups/guide.html

        In the future, please choose a single newsgroup to post your questions
to.

        I'm setting follow-ups to .bugdiscussion.  I'd appreciate it if anyone
else responding to to this thread would do the same.

        I very much doubt the leak is in IB -- it could be in InterClient or
your code, but I don't know enough about IC and Java to say which one.

        -Craig

--
Craig Stuntz (TeamB)       Senior Developer, Vertex Systems Corp.
Delphi/InterBase weblog:   http://delphi.weblogs.com
Use Borland servers; posts via others are not seen by TeamB.
For more info, see http://www.borland.com/newsgroups/genl_faqs.html

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Well, I'll have to sign-off here then as I don't use
interserver/interclient. But, I strongly suspect it is something in that
layer that is the root cause of the problem. The IB API is very solid and
its few know bugs do not manifest themselves in the way you are describing.

Does your application need to do what you are doing here which kills the
server or are you just proud of the fact (as I would be) to have come up
with some simple code that crushes the server?

Regards,
Jason Wharton
CPS - Mesa AZ
http://www.ibobjects.com

Quote
"Chris Smith" <csm...@tripwire.com> wrote in message

news:3b37a431$1_1@dnews...
Quote
> Actually, I tried that. Unfortunately, the ibserver.exe process does not
> recover the memory.

> --Chris.

> "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> news:3b37a3d0$1_1@dnews...
> > Then it is for sure a bug in the interserver/interclient driver. Stop
and
> > start the interserver process and I bet within seconds the ibserver
> process
> > recovers handsomely.

> > --
> > Jason Wharton
> > CPS - Mesa AZ
> > http://www.ibobjects.com

> > "Chris Smith" <csm...@tripwire.com> wrote in message
> > news:3b37a005$1_1@dnews...
> > > I don't think it is tied at all to Java garbage collection. When the
> test
> > > program exits, all of the memory is still consumed by the IB
processes.

> > > --Chris.

> > > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > > news:3b379e24$1_2@dnews...
> > > > I would suspect this as a bug in the interserver/interclient
> > > implementation
> > > > then.
> > > > What you are doing is chewing up statement handles and for some
reason
> > > they
> > > > are not being deallocated.
> > > > Is there any way you can know if garbage collection is taking place.
> > > > I sure would hate to think that database resources are being tied to
> > JAVA
> > > > garbage collection.
> > > > There should be a way that you can explicitly demand database
> resources
> > to
> > > > be released before an object is put on the garbage collection queue.

> > > > Sure glad I use Delphi (and not IBX).

> > > > FWIW,
> > > > Jason Wharton
> > > > CPS - Mesa AZ
> > > > http://www.ibobjects.com

> > > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > > news:3b379a7d$1_2@dnews...
> > > > > Actually both processes are showing the leak (ibserver.exe and
> > > > > interserver.exe). As far as not unpreparing goes, the only method
> > > > available
> > > > > is close() to deallocate. Here is the Javadoc:

> > > > >  /**
> > > > >   * Releases this <code>Statement</code> object's database
> > > > >   * and JDBC resources immediately instead of waiting for
> > > > >   * this to happen when it is automatically closed.
> > > > >   * It is generally good practice to release resources as soon as
> > > > >   * you are finished with them to avoid tying up database
> > > > >   * resources.
> > > > >    * <P><B>Note:</B> A <code>Statement</code> object is
> automatically
> > > > closed
> > > > > when it is
> > > > >   * garbage collected. When a <code>Statement</code> object is
> closed,
> > > its
> > > > > current
> > > > >    * <code>ResultSet</code> object, if one exists, is also closed.
> > > > >    *
> > > > >    * @exception SQLException if a database access error occurs
> > > > >    */

> > > > > Also, note that it does not leak memory when I replace "Statement"
> > with
> > > a
> > > > > "PreparedStatement". Therefore, I am guessing that my code is
> correct.

> > > > > --Chris.

> > > > > "Jason Wharton" <jwhar...@ibobjects.com> wrote in message
> > > > > news:3b37991b$1_2@dnews...
> > > > > > Which process is showing the memory growth?

> > > > > > IBServer or the server for Interserver that hosts Interclient?

> > > > > > You are creating and allocating a statement and merely closing
it
> > > (which
> > > > > > relates to a cursor only). How can it actually unprepare and
> > > deallocate
> > > > > the
> > > > > > statement? Apparently JDBC expects you to explicitly handle this
> > > detail.

> > > > > > I'm a Delphi guy so I could be wrong... but I am very confident
> your
> > > > code
> > > > > is
> > > > > > wrong.

> > > > > > Regards,
> > > > > > Jason Wharton
> > > > > > CPS - Mesa AZ
> > > > > > http://www.ibobjects.com

> > > > > > "Chris Smith" <csm...@tripwire.com> wrote in message
> > > > > > news:3b37834f$1_2@dnews...
> > > > > > > Over the weekend, I had the pleasure to find the location of
an
> > > > enormous
> > > > > > > memory leak in Interbase. I searched the newsgroups, and other
> > > threads
> > > > > > > related to IB, and found several people complaining about such
> > > leaks.
> > > > > > > However, all of the responses from Borland folks have stated
> that
> > > the
> > > > > > issue
> > > > > > > was due to unclosed connections, etc.

> > > > > > > Well, I have a reproducible test case that is only a couple
> lines.

> > > > > > > My configuration is as follows:

> > > > > > > Interbase 6.01 (Open Source)
> > > > > > > Interbase Client 1.60 (I have also reproduced with 2.0) (Open
> > > Source)
> > > > > > > JDBC Drivers 1.60
> > > > > > > Windows 2000

> > > > > > > The problem appears to show itself when using JDBC Statement
> > > > operations
> > > > > > that
> > > > > > > perform updates (delete, update, etc.). Here is the sample
code
> > that
> > > > > will
> > > > > > > allow you to deplete your computer of all its memory.

> > > > > > > try
> > > > > > > {
> > > > > > >     Connection con = /* Create JDBC connection here using
> > > > > > > "interbase.interclient.Driver" */
> > > > > > >     for( int i = 0; i < 1000; i++ )
> > > > > > >     {
> > > > > > >         int newVal = (int)(Math.random() * 100000);
> > > > > > >         Statement st = con.createStatement();
> > > > > > >         st.executeUpdate( "update foo set foocol = " + newVal
+
> "
> > > > where
> > > > > > > foocol > 0" );
> > > > > > >         st.close();
> > > > > > >         con.commit();
> > > > > > >     }
> > > > > > >     con.close();
> > > > > > > }
> > > > > > > catch( Exception e )
> > > > > > > {
> > > > > > >     e.printStackTrace();
> > > > > > > }

> > > > > > > The DDL is the following:    create table foo( foocol
integer );

> > > > > > > I insert a row before testing, but it doesn't matter, the leak
> > > occurs
> > > > in
> > > > > > > either case.

> > > > > > > When you run this program, watch the memory consumption of
> > > > ibserver.exe
> > > > > > and
> > > > > > > interserver.exe (the connection). Both of these processes will
> > leak
> > > > > memory
> > > > > > > at a huge rate. When the program exits, none of the memory is
> > > > reclaimed.

> > > > > > > The most interesting part is that if you change the Statement
> > above
> > > to
> > > > a
> > > > > > > PreparedStatement, the processes hardly leak at all. If they
do
> > leak
> > > > it
> > > > > is
> > > > > > > at a very low rate.

> > > > > > > If there is anything I am missing, or something I am doing
> wrong,
> > > > please
> > > > > > let
> > > > > > > me know.

> > > > > > > --Chris.

Re:HUGE MEMORY LEAK in Interbase makes product completely unusable


Quote
"Chris Smith" <csm...@tripwire.com> wrote in message

news:3b379654$1_2@dnews...

Quote
> Well, if you look at the code that I have with the original post, the
> transaction is only one update statement. Also, the table can be
completely
> empty, and the memory is still leaked. Thus, I don't think it has anything
> to do with data page size, etc. Also, memory is leaked from the connect
> (interserver.exe), which probably wouldn't be related to the page size.

For Interbase, it will still happily allocate memory up to the limit you
set. Assuming 4K page size, you should set max pages to no more than 10,000.
This will allow IB to use up to 40 meg - but no more - for caching (more
than 40 meg results in a performance "knee"). I cannot speak for interserver
as I haven't used it, but I've never seen a problem with IB eating all
memory.

--
Wayne Niddery (Logic Fundamentals, Inc.)
RADBooks: http://www.logicfundamentals.com/RADBooks/delphibooks.html
"Bandwagons are like streetcars, there'll be another along in a few
minutes" - Me!

Go to page: [1] [2]

Other Threads