pgsql-5196

Version:

8.3.8

Bug Link:

http://postgresql.1045698.n5.nabble.com/BUG-5196-Excessive-memory-consumption-when-using-csvlog-td2130188.html

Symptom:

When using csvlog instead of stderr(can be configured in postgresql.conf with log_destination = 'stderr') , the postgres logger process consumes a lot of memory.

How it is diagnosed:

Source code and bug report analysis

Root Cause:

There’s memory leak in syslogger: logfile_rotate() would leak a copy of the output filename if CSV logging was enabled and only one of the two possible output files got rotated during a particular call (which would, in fact, typically be the case during a size-based rotation). This would amount to about MAXPGPATH (1KB) per rotation.

/*

 * perform logfile rotation

 */

static void

logfile_rotate(bool time_based_rotation, int size_rotation_for)

{

        … ...

        if (time_based_rotation)

        {

                filename = logfile_getname(next_rotation_time, NULL);

                if (csvlogFile != NULL)

                        csvfilename = logfile_getname(next_rotation_time, ".csv");

        }

        else

        {

                filename = logfile_getname(time(NULL), NULL);

                if (csvlogFile != NULL)

                        csvfilename = logfile_getname(time(NULL), ".csv");

        }

if (time_based_rotation || (size_rotation_for & LOG_DESTINATION_STDERR))

{

        …

        /* When destination is not stderr, this path is not executed! */

        if (filename)

                pfree(filename);

        if (csvfilename)

                pfree(csvfilename);

        return;

}

...

        /* Same as above, but for csv file. */

        if (csvlogFile != NULL &&

                (time_based_rotation || (size_rotation_for & LOG_DESTINATION_CSVLOG)))

        {

                ...

                if (!fh)

                {

                        int                        saveerrno = errno;

                        ereport(LOG,

                                        (errcode_for_file_access(),

                                         errmsg("could not open new log file \"%s\": %m",

                                                        csvfilename)));

                        ...

-                        pfree(csvfilename);

+                        if (filename)

+                                pfree(filename);

+                        if (csvfilename)

+                                pfree(csvfilename);

                        return;

                }

...

+        if (filename)

+                pfree(filename);

+        if (csvfilename)

+                pfree(csvfilename);

        set_next_rotation_time();

}

/*

 * construct logfile name using timestamp information

 *

 * If suffix isn't NULL, append it to the name, replacing any ".log"

 * that may be in the pattern.

 *

 * Result is palloc'd.

 */

static char *

logfile_getname(pg_time_t timestamp, const char *suffix)

{

        char           *filename;

        int                        len;

        filename = palloc(MAXPGPATH);  //#define MAXPGPATH                1024

        ...

}

Failure type:

Memory leak

Is there any log message?

No.

Can ErrLog insert a log message?

Yes. Malloc sampling.