I am setting up the Scala Stream collector, but I am confused with the setup instructions given.
Where will I be configuring and running the scala stream collector? Will it be on EC2 or somewhere else?
Thanks.
Yes - you can configure and run the Scala stream collector on one (though preferably more than one for HA) EC2 instances.
I’ve used elastic beanstalk w/o RDS or SQS. Packed binaries and config files into a zip archive ( per beanstalk manual ),exposed services through Procfile, set load balancing and HA options and pretty much it. Has been very stable.
Watch out for misconfiguration (e.g. kinesis message size can’t exceed 500 events ), otherwise check the logs, run a load test and you’re good to go. Will take 20-30 minutes or so start to finish.
Thanks. in the config.hocon.sample file, it is asking for interface and port, would this be the public IP of my EC2 instance? and how do I find out which port my EC2 is running on?
The interface you can leave as “0.0.0.0” and the port is the port that the stream collector will run on. The port can be pretty much anything you want (though typically ports are 80 or 8080) - you’ll just need to ensure that your load balancer is forwarding HTTP(S) traffic to the port on those instances.