Vertica on AWS
Hi,
I have few questions regarding using Vertica on AWS. I am thinking of storing 1 TB of data.
1. What are the minimum instance type required? Is it c4.8xlarge? Can I select anything lower than that?
2. Do I need to have 3 instances running or is 1 or 2 instance is enough?
3. Having 3 c4.8xlarge a month would cost more than $4000 a month, any cheaper solution on AWS?
Thanks,
Raj
0
Comments
If you are using Vertica build AMI yes you are limited to thos huge instances.
You can use any of thier instance for a single node cluster.
For a 3 + cluster you will need instances that accept Placement Groups, (but are cheap as well).
I am running Vertica on AWS also for my DEV.
If you want a ready image of a Vertica node i can provide you with.
So can we just run 1 c4.8xlarge instance on AWS and have 1 TB of data on it?
Sure you can.
But just to test it orin the preparation phase you can use even a t2.micro instance.
Once the node is ready to be deployed just upgrade your instance to a better bigger one
.
Unless you need a c4.8xlarge (1290$/month) use a t2.micro(9 $/month) for your testing.
You can have as much space as you want , important to know that AWS will charege you for the data out and not data in. Which is great for analitical stuff where you pump huge amounts of data and get a bit on information.
Like i said those c4.8xlarge instances are used by Vertica AS template so oyu could get optimized performance but is not the case when you look at the costs. And instead of scaling-up (use strong machines) you cna scale-out(use small machines and alot of them) this will allow you a more MPP env and also there is a more complex task at hand for you manage.
Here is a usefull tool to messure your instance costs
http://calculator.s3.amazonaws.com/index.html
When I select from "Amazon Machine Image (AMI)", I see that only c4.4xlarge or higher is enabled and other are disabled (I cannot select them).
How can I use lower cost options? I am interested in using Vertica on AWS.
Thank you for suggestion.
Hi,
Like i told you , that template that you use wa sbuild by Vertica team and it only has 2 options of instances.
The template is great as it comes with the configures you ca ask for.
But you can easy manage to build your own template and if this is out of your hands go ahead and build am AMI in linux to have all the prereqs of instaling vertica install the rpm on it and with it create how many node you want.
Even better i will make a video and posted for you on youtube, i already have some AMI ready to go.
You just need to follow the video
.
Follow this link to https://www.youtube.com/watch?v=7n7YYCvKHXc
If you like it give it a like if not ask for improvements
.
We (HP Vertica) restrict the official AMI to higher instance types because lower instances have poor stability under load. Vertica uses spread (which uses udp point to point) for control communication. Lower instance types do not have adequate resources in terms of network bandwidth/latency to maintain needed response times on the control network.
We have seen instances of users using lower end instance types where nodes drop out of the cluster due to resource starvation resulting in database instability.
Although its trivial to build your own AMI given our installer, we recommend you stay with the published AMI for the reason mentioned above. Offcourse if you are using it for development or testing rolling your own AMI is acceptable way or reducing costs. However for production uses you will find that lower end instances do not provide a consistent and predictable user experience.
furthermore, if you contact vertica support with performance or stability issues and are using a "unofficial" AMI, we might not be able to help you unless you move to a supported instance type with the recommended configuration (i.e. equivalent of official AMI published on Marketplace from time to time).
Hope this helps
Hi Raj
1. What are the minimum instance type required? Is it c4.8xlarge? Can I select anything lower than that?
> If you use the official AMI, unfortunately you are restricted to the instance types listed.
2. Do I need to have 3 instances running or is 1 or 2 instance is enough?
> you only need more than 1 instance if you need your cluster to be ksafe (i.e. you need fault tolerance and/or high availability). offcourse this helps with distribution of load as well and having more nodes in most cases improves performance.
3. Having 3 c4.8xlarge a month would cost more than $4000 a month, any cheaper solution on AWS?
> For development and testing you can roll your own AMI, but for production uses i recommend the one on marketplace.
> if you use reserved instances, AWS may provide a a better rate.
Thanks,
Raj
skeswani , performance will not be optimum and also don`t use this AMI i have made public to run production stuff. If you choose to do it "You are on your own!!!".
The whole purpose of this AMI is to make it easier to run dummy instances of Vertica where you can test your stuff or paly and learn how Vertica is doing work.
And if you consider buying Vertica license make sure you get the right support from HP Vertica to start your enviroment on AWS, own infra or any cloud.