My stream collector process dead randomly. I can see process run in background and port be listening, but it lost response randomly. I need restarrt it to make it works again.
[proxy_http:error] …Connection reset by peer: … AH01102: error reading status line from remote server 123.123.123.123:80, referer: https://abc.abc.com/
I try check http active from localhost. When collector works,checking message like this:
HTTP WARNING: HTTP/1.1 400 Bad Request - 185 bytes in 0.057 second response time |time=0.056884s;;;0.000000 size=185B;;;0
When collector stop working,checking message like this:
HTTP CRITICAL - No data received from host
when crash happened,collector script out put like this:
INFO: Sleeping …
Feb 16, 2017 1:28:52 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:29:02 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:29:13 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:29:23 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 14 datums.
Feb 16, 2017 1:29:33 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:29:43 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:29:52 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: No activities assigned
Feb 16, 2017 1:29:52 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: Sleeping …
Feb 16, 2017 1:29:54 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:30:04 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:30:14 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 14 datums.
Feb 16, 2017 1:30:24 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:30:34 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:30:45 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:30:53 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: No activities assigned
Feb 16, 2017 1:30:53 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: Sleeping …
Feb 16, 2017 1:30:55 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:31:05 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:31:15 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 14 datums.
Feb 16, 2017 1:31:26 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:31:36 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:31:46 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:31:54 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: No activities assigned
Feb 16, 2017 1:31:54 AM com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker info
INFO: Sleeping …
Feb 16, 2017 1:31:56 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:32:07 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:32:17 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 14 datums.
Feb 16, 2017 1:32:27 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:32:37 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
Feb 16, 2017 1:32:47 AM com.amazonaws.services.kinesis.metrics.impl.CWPublisherRunnable publishMetrics
INFO: Successfully published 4 datums.
AWS Collector Kinesis Stream monitor show no active during crash time
There is a number of possible issues. Most popular ones are not sufficient ram memory, consumed all the cpu credits (we noticed it on EC2 and container service on AWS). Another possibility is a proxy/load balancer or another intermediate breaking your requests.