OT: VM slicing and dicing

Mailman List Mirror (Read Only)
1

Hey gents:

As always I value your input. Best resource on the planet! =)
I'm hoping this isn't too off-topic if so please respond to me offline if so.

I figured since most of everyone here are operators working in a datacenter, you may or may
not have experience with virtualization software that allows you to configure VM's on the fly.

I'm not looking for companies that offer this service, but the actual software engines that allow you
to create VM's on the fly. So a customer goes to your website and says I want Win2008 with 8gigs of RAM and 120gigs of HDD.
Just like custom configuring a new PC.

Does anyone here have experience or knowledge of companies that offer this type of software engine?

Thanks in advance!

Brandon

2

I'm a big fan of Citrix's XenServer system - I've only created VMs using their XenCenter software, but from what I've heard, their API is easy to work with.

Tim Burke
tb@tburke.us
815.556.2000

3

if you are using KVM (or even VMware) and you can write shell scripts, you can do this in house. both have the ability to create VMs from the command line. in KVM you can create a VM with a one liner.

-g

4

We've been looking at Cisco's Unified Computing System (UCS) blade
server, which appears to have great potential. Very fast, and eliminates
almost all top-of-rack copper cabling from servers to top-of-rack
switch. Custom-built for VMWare optimization, but other virtualization
OS's will run also from what I have read. Ten GiGE and FCoE are the
entry points at the server access layer.

5

no copper cables 10G and FC is all you need to deploy images. 8)
-g

6

Hello:

I figured since most of everyone here are operators working in a datacenter, you may or may
not have experience with virtualization software that allows you to configure VM's on the fly.

I'm not looking for companies that offer this service, but the actual software engines that allow you
to create VM's on the fly. So a customer goes to your website and says I want Win2008 with 8gigs of RAM and 120gigs of HDD.
Just like custom configuring a new PC.

You might want to check out openQRM which is an open source project that can do this type of dynamic provisioning, as well as other datacenter management features, for multiple virtualization technologies (VMware, Xen, KVM, etc). It can also handle provisioning of physical machines and V2P and P2V migrations.

     http://www.openqrm.com/?q=node/2

There's also an enterprise version so that you can get support and services if needed:

     http://www.openqrm-enterprise.com/

HTH,

Aaron

7

Thanks everyone for your input today on this topic. I wanted to recap with a list of sites that everyone has suggested
both online and offline for FYI purposes.

http://www.vmware.com/products/vcloud-director/

http://www.microsoft.com/systemcenter/en/us/default.aspx

http://cloud.com

http://www.gogrid.com/

http://www.digitalmines.com

http://www.proxmox.com/products/proxmox-ve

http://www.openqrm-enterprise.com/

http://www.openstack.org/

8

Thanks everyone for your input today on this topic. I wanted to recap with a list of sites that everyone has suggested
both online and offline for FYI purposes.

http://www.microsoft.com/systemcenter/en/us/default.aspx

I haven't used system center, but have been very happy with Microsofts other management offerings. In particular the combination of WMI and Active Directory is pretty slick. Now days with W2k8 Server Core and VM friendly licensing, the Microsoft OS density on a hardware node is starting to approach Linux density levels.

http://www.proxmox.com/products/proxmox-ve

I use Proxmox exclusively and am very happy with it. It's a great product. You might need to do a bit of CLI work if you want to support multiple VLANS or other slightly advanced features. I'm lazy but I might get around to patching the web UI at some point to support the stuff I do manually. The OpenVZ docs are very clear and the process is pretty trivial to do on the CLI.

http://www.openqrm-enterprise.com/

This has received some serious attention from me, but it seemed a bit heavy on the startup requirements and it wanted to own my entire infrastructure. Proxmox was just plug and play and reduced the effort to deploy virtual machines. Anyone here using openqrm? How demanding is it? Can you just utilize the pieces you want? These days most users have existing systems in place to handle storage, security, monitoring, os configuration management etc. I guess if you are a completely new startup, then OpenQRM might make sense.

http://www.openstack.org/

Ah yes. The new comer of sorts. Anyone looked at this in detail? Beta deployed it?

9

I've used OpenVZ at many sites and been really happy with it.

Managing OpenVZ from the CLI is easy. I wrote wrapper scripts to perform the desired functions.

It has extensive documentation available. From a documentation point of view it really stands out among OSS and even commercial apps.

Cheers,

Rob

10

I use Proxmox exclusively and am very happy with it. It's a great product. You might need to do a bit of CLI work if you want to support multiple VLANS or other slightly advanced features. I'm lazy but I might get around to patching the web UI at some point to support the stuff I do manually. The OpenVZ docs are very clear and the process is pretty trivial to do on the CLI.

Managing OpenVZ from the CLI is easy. I wrote wrapper scripts to perform the desired functions.

Yeah. It's very easy. Proxmox is for super lazy people like me. :slight_smile:

It has extensive documentation available. From a documentation point of view it really stands out among OSS and even commercial apps.

Yes. The documentation is fantastic. Top notch. OpenVZ is very simple and utilizes existing features in Linux directly. As opposed to XEN (at least as it ships with centos 5) which utilizes an entire super structure of complex shell scripts to do it's networking setup. If you have a few years of server admin experience it's very easy to get up and going. You can utilize all your existing CLI knowledge.

11

Make sure you have sized the beancounters (resource limits) for your
applications and workload. With the default beancounter settings, a
default centos5 install of postfix (100 process max -- per service, ie
100 smtpd and 100 smtp) or dovecot (100 each for imap/pop3, etc.) you
will run into limits very quickly.

Using vzsplit helps a lot.

12

I know VMWare has something that allows you to do what you are asking
(vSphere I believe but I stand to be corrected). Though I am not so sure
about the "on the fly" as any Vm enviroment requires careful planning before
starting to add the VMs to your hardware

Regards
Raymond Macharia

13

Brandon,
It really depends on the hypervisor in operation. You can take a look at
vCloud Director (http://www.vmware.com/products/vcloud-director/) and
BMC
(http://www.bmc.com/products/product-listing/bmc-cloud-lifecycle-management.html)

14

Thanks guys for keeping this topic alive. =)

I'm leaning towards the opensource or at least the Xen side of things. I haven't yet fully evaluated vCloud Director but I get the gut
feeling that anything "VMware" is going to be costly.

Is that a fair assumption?

The issue is that I'm looking for an application that is as turnkey as possible, even if it's a little bit more. That "could" be vCloud Director, I don't know yet....

But I do know that if we have to invest in writing a lot of custom scripts to get what we want, then we don't have the resources for that....

15

How about I send you some terms to search for, using your favorite
search engine...
Multi-Tenant Hosting > Cloud Computing > IaaS / HaaS
(Infrastructure as a Service) > Self-Service Provisioning
Because the question is so vague, I think you need more research.
If you read the documentation of portal software, you should be able
to tell to what extent it would be "turn key"

Before looking too closely at any offering... some things to think about are..
How would you go about handling virtual networks and access to them?
Will you want one shared network (with requisite Layer 2 security minefield),
or will your portal of choice somehow decide to permission and make
certain LANs available to certain users' VMs?

There will be security and performance considerations that some portal
software programs allow you to answer, and some do not. So you
need to decide the hard requirements for security, management
flexibility, UI attractiveness/ease of use, functionality for the
end user, resource management, and price :slight_smile:

Different portals have different options, so define requirements first.
A Multi-Tenant IaaS environment (meaning different users sharing
pieces of metal, storage, etc) brings in some complexity.

Think about how will the resources be balanced? E.g. Will you have a portal
place workloads on its own, or rely on some outside system like vmware DRS.
Will the portal implement and enforce resource SLAs for Network latency/loss,
limit the number of VMs per NIC or per datastore, Memory, CPU
and provide I/O response delay assurances, or will machines be left
underutilized
/ overutilized, because the portal is bad at optimizing placement on physical
servers, or bad at avoiding overcommit?

For an IaaS provider, underutilization eventually means you are eating
more kW·h than necessary, and overutilization could be
immediately detrimental.

The different major virtualization software vendors each have their own
Self-Service Provisioning solutions, and there are some third party programs.
Most are for Enterprise internal self-provisioning; Hosting providers
might have
special requirements like "integrated user signups and billing"
and "no license restriction against provisioning for outside users".
I would expect these to be more expensive, or include monthly per-user fees.

Offhand I recall Virtuozzo [perhaps the oldest?], Enomaly /
Enomalism, enStratus, MS Dynamic Datacenter Kits which are a
framework, VMware vCloud Express through the VSPP, Citrix XCP,
Eucalyptus, as interesting
by no means exhaustive.

16

Thanks for the suggestions James! One of the issues I had, (which is why I turned to NANOG) was that I wasn't entirely
sure what keywords to search for!! So thank you for that. All of the criteria's you brought up are valid and I will add them
to the list of things to consider.

It's awfully difficult to figure out who can do what as it's just not possible to test all the different vendors out there unless
you have a large R&D team and a lot of time.

I think we are on the same page as far as what "We" think I need. But just to clarify.

1) We'd like to be able to have a web portal where new or existing clients could request servers of all types: windows, linux etc...
Configure what it is that they need and in some amount of time, the VM's are provisioned. They receive some kind of email confirming
that their new provisioned server is available.

2) Backend - Since we haven't invested much time into the backend, we're open to all possibilities. It doesn't need to be VMware at all.
Xen seems to be extremely popular.

3) Licensing - Of course this will be all unique to each vendor but the more complicated the licensing, the more it's a turn off and difficult to
keep track of. Not to "plug". But so far OnApp's pricing is very straightforward.

4) Multi-Tenant - Absolutely needs to support this.

I don't expect anyone here to do research for me, but I assume that being a network operator, many of us would have some input and clearly
I've received great feedback. I've been in touch with numerous vendors that were given to me from this thread and I can't wait to demo/try their products....

One question I do have for any that actually read through this entire email (haha) is about the physical network switch. Is there a case for the switch, especially
in today's high density environment to go with 1GIG switches as the minimum? It seems pretty obvious but I'm wondering if it's really a necessity?
Can anyone on this list argue that 10/100 will be suffice?

Thanks again!

Brandon

17

The issue is that I'm looking for an application that is as turnkey as possible, even if it's a little bit more. That "could" be vCloud Director, I don't know yet....

Hi Brandon. Turnkey is a relative term - relative to the experience and knowledge level of those operating the system. I've used a lot of virtualisation systems and these days consider most of them pretty much turnkey (commercial and OSS alike). Really, you install them, setup networking and install some virtual boxes. Options like OpenVZ come with precooked images that you can start with so you don't even need to do an install if you don't want to.

Anything as complicated as virtualisation is always going to have some learning curve though.

But I do know that if we have to invest in writing a lot of custom scripts to get what we want, then we don't have the resources for that....

Most of the virtualisation systems I've setup are operated by the sysadmins from the cli. I often write custom scripts to wrapper functions but in reality these are often very short, sometimes just a few lines long.

BTW I think this would be a perfect topic for a SAGE or LOPSA list. If you're involved in this sort of work then you may wish to consider joining one or both of these organisations and participating in the lists.

Cheers,

Rob

18

1 GiGE switches at a minimum; some vendors (e.g., arista) have low cost
48 port 1000/10000 switches. Cisco's UCS system uses 8 10 GiGE uplinks
where the servers (running a hypervisor kernel) plug into a chassis
backplane with 2 10 GiGE connectors each, that mux 10 GiGE and 4/8/16
GiG FC over the combined 80 Gig uplinks. Think about latency, not just
bandwidth. 100 Mb is 100 times slower in serialization/deserialization
of bits on/off the wire. Also, do you really want the cable management
issues associated with multiples of 48 copper cables from servers to
top-of-rack switches?