[redis]github-91 Report

https://github.com/antirez/redis/issues/91

1. Symptom

Memory leak can freeze Redis from accepting new client requests.

1.1 Severity

Critical

1.2 Was there exception thrown?

Yes. Error: Error: ERR command not allowed when used memory > 'maxmemory'

at Command.callback (/home/trellis/trellis/node_modules/redis/index.js:774:27)

at RedisClient.return_error (/home/trellis/trellis/node_modules/redis/index.js:382:25)

at RedisReplyParser. (/home/trellis/trellis/node_modules/redis/index.js:78:14)

at RedisReplyParser.emit (events.js:64:17)

at RedisReplyParser.send_error (/home/trellis/trellis/node_modules/redis/lib/parser/javascript.js:265:14)

at RedisReplyParser.execute (/home/trellis/trellis/node_modules/redis/lib/parser/javascript.js:124:22)

at RedisClient.on_data (/home/trellis/trellis/node_modules/redis/index.js:358:27)

at Socket. (/home/trellis/trellis/node_modules/redis/index.js:93:14)

at Socket.emit (events.js:64:17)

at Socket._onReadable (net.js:672:14)

1.2.1 Were there multiple exceptions?

Well, the above msg appears many times.

1.3 Scope of the failure

single client

2. How to reproduce this failure

2.0 Version

2.0.4

2.1 Configuration

maxmemory enabled

2.2 Reproduction procedure

1. Introduce a network error btw slave & master so it won’t consume the data from master fast enough  (disconnect)

2. client keeps writing to the server to produce more data

2.2.1 Timing order

in this order

2.2.2 Events order externally controllable?

yes

2.3 Can the logs tell how to reproduce the failure?

Yes

2.4 How many machines needed?

2 (master + slave)

3. Diagnosis procedure

3.1 Detailed Symptom (where you start)

Error log and freeze, also gives you the stack trace.

3.2 Backward inference

From the log we know stack trace and the mem usage is big. It also shows:

client_longest_output_list:4490

Indicates the mem consumption might be from the output buffer.

4. Root cause

The output buffer has mem leak.

4.1 Category:

Semantic