Advanced Nutanix: ILM Hacking pt. 1


In the post I’ll explain what I do to start with a fresh Nutanix environment during test runs as well as other tips and tricks to get ILM to work as you want for your testing.  NOTE: Stay tuned, I have another post going deep into what we’re doing with ILM and its awesomeness!

A great deal of what I do on a daily basis focuses around bench marking various applications and workloads on the Nutanix platform.  In doing so, I always want to start with a “fresh” environment, however it’s not always possible to destroy the cluster in an existing environment with VMs and data.  One way I get around this is by forcing the SSD tier to completely flush to HDD which can be achieved using the methods below.

Curator Gflags

Gflags are an advanced configuration parameter which allows us to modify configuration values on the Nutanix platform.  DISCLAIMER: Any modification of Gflags should only be performed by a Nutanix SE or a Nutanix Certified Professional and should not be modified on any production systems nor necessary to do so!

Here are the Curator Gflags we’re going to be working with:

  • Explanation: This Gflag specifies the threshold percentage for SSD tier utilization at which will force an ILM down-migration of cold data when is breached.
  • Default: 75
  • Explanation: This Gflag specifies the amount of cold data that will be moved during an ILM down-migration
  • Default: 15

NOTE: My fellow colleague Jerome Joseph (@Getafix_Nutanix) has another great post on curator modification which can be found here: LINK

This is a common thing I do when I want to start with a fresh and completely empty SSD tier.  The following will allow you to force the full SSD tier to flush to HDD.  NOTE: Don’t attempt this in a production environment as you want to keep your existing data where it is!

  1. Log into a Nutanix Controller VM
  2. Set the curator_tier_usage_ilm_threshold_percent to 5% (This will cause a migration to take place over 5% utilization)
  3. Set the curator_tier_free_up_percent_by_ilm to 95% (This will essentially flush the full SSD tier)
  4. Navigate to http://<Curator Master>:2010/master/control and start a full scan
  5. Watch the jobs status on the Curator Master page
  6. The available SSD capacity will be 95% of the total available
  7. Set the curator_tier_usage_ilm_threshold_percent to 95% (This will make sure a ILM down-migration doesn’t occur until SSD is 95% full)
  8. Run your tests (eg. Bonnie++, IOmeter, IOzone, etc.)

Here’s a way to do this all programmatically from the CLI on a CVM:

Stay tuned, I’ll have another post shortly that will cover the goodness we’re doing with ILM!


Legal Mumbo Jumbo

Copyright © Steven Poitras, The Nutanix Bible and, 2014. Unauthorized use and/or duplication of this material without express and written permission from this blog’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Steven Poitras and with appropriate and specific direction to the original content.