httpd-43504

Version:

2.2.6

Bug Link:

https://issues.apache.org/bugzilla/show_bug.cgi?id=43504  

Symptom(Failure):

There are two servers which are communicating via mod_proxy / balancer and AJP. One is primary and the other standby. Normally the first gets served. After upgrading to 2.2.6 the state of the primary changes to error and the standby does not jump in. When accessing the webpage got “Service Temporarily Unavailable”. The error log message states an AJP buffer overflow.

[Fri Sep 28 14:38:53 2007] [error] ajp_msg_append_uint8():
BufferOverflowException 4 4

How it is diagnosed:

We did not reproduce the failure, rely on source code analysis.

Root Cause:

AJP message  buffer is not initialized big enough (wrong SIZE macro was used!)

(mod_proxy_ajp.c)

apr_status_t ajp_msg_append_uint8(ajp_msg_t *msg, apr_byte_t value) {

   apr_size_t len = msg->len;

   /*** Good practice: Before dereferencing a buffer, first do some bound check. **/

   if ((len + 1) > msg->max_size) {

     // since len is well maintained, there must be something

     // wrong with max_size

     // the max_size is set when creating the ajp message

     // ERROR: ajp_log_overflow will print the error message

       return ajp_log_overflow(msg, "ajp_msg_append_uint8");

   }

   msg->buf[len] = value;

   msg->len += 1;

   return APR_SUCCESS;

}

int ajp_log_overflow(ajp_msg_t *msg, const char *context) {

   ap_log_error(APLOG_MARK, APLOG_ERR, 0, NULL,

                "%s(): BufferOverflowException %" APR_SIZE_T_FMT

                " %" APR_SIZE_T_FMT,

                context, msg->pos, msg->len);

   return AJP_EOVERFLOW;

}

apr_status_t ajp_msg_create(apr_pool_t *pool, apr_size_t size, ajp_msg_t **rmsg) {

   ajp_msg_t *msg = (ajp_msg_t *)apr_pcalloc(pool, sizeof(ajp_msg_t));

   ...

   msg->len = 0;

   msg->header_len = AJP_HEADER_LEN;

   msg->max_size = size;

   *rmsg = msg;

   return APR_SUCCESS;

}

/**By tracing call graph or some domain knowledge, we can get ajp_handle_cping_cpong*/

apr_status_t ajp_handle_cping_cpong(... ...) {

   … ...

   ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, r->server,

                        "Into ajp_handle_cping_cpong");

  // AJP_HEADER_SZ_LEN is only 2!

  // ajp.h +#define AJP_PING_PONG_SZ            128

-   rc = ajp_msg_create(r->pool, AJP_HEADER_SZ_LEN+1, &msg); (Fault!)
+  rc = ajp_msg_create(r->pool, AJP_PING_PONG_SZ, &msg);

   if (rc != APR_SUCCESS) {

       ap_log_error(APLOG_MARK, APLOG_ERR, 0, r->server,

              "ajp_handle_cping_cpong: ajp_msg_create failed");

       return rc;

   }

   …

}

Is there any log message?:

Yes! Bound check (memory safety check). Helpful for diagnosing the bug. A good practice to learn.

Can Errlog anticipate this log msg?

Yes. Memory safety check!!!

  if ((len + 1) > msg->max_size) {

       return ajp_log_overflow(msg, "ajp_msg_append_uint8");

   }

   msg->buf[len] = value;