Question

Theo Ekelmans on Tue, 24 Feb 2015 22:34:04


Hi All,

In order to get to the size and free space info on mountpoints i used several variations on this theme:

EXEC master..xp_cmdshell 'wmic logicaldisk where (drivetype ="3" and volumename!="RECOVERY" AND volumename!="System Reserved") get deviceid,volumename  /Format:csv'

and then decode what came back, not pretty, but hey... it worked and was reasonably fast, about  100-400ms or so.

But due to some weird DR STRETCHED SAN config fubar of our hosting supplier we found ourselves unable to expand mountpoints on the DR stretched 3 way cluster, and each time when we ran out of space we simply piled on more and more mountpoints and files to databases.

Don't shoot the messenger guys....   i just inherited this setup and have to deal with it until it reaches it's intended lifespan.

As the mountpoint number went up, the runtime of the xp_cmdshell / wmic took longer and longer to complete, up to several minutes, and i started to worry :(

So convinced something had to be done, i grabbed a C# book and did some serious catching up on my aging 30 year old C knowledge (yeah, i still have the white/blue kernighan and richie book from my school days), and wrote my first CLR. 

After the struggling with hello world issues that come with using Visual Studio 2013 C# for the first time, i got a cmd window working with precisely what i needed. But only to find out that .NET SQL CLR is "some what limited" in the things you can use.

Long story short, (and with many a thanks to all who share code snippets) this is what i came up with; a CLR that does 3 things: 
- Physical disk info (no mountpoint support)
- Logical disk info (with mountpoint support) 
- and a SPLIT function.

Here's my code (download)

----------------------------------------------------------------------------------------------------------------------
using System;
using System.Collections; /// IEnumberable
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using System.Diagnostics;
using System.IO; /// DriveInfo
using Microsoft.SqlServer.Server;

public partial class StoredProcedures
{

    //-------------------------------------------------------------------------------------
    // Logical volume info (works for mountpoints)
    //-------------------------------------------------------------------------------------
    /*
    AvailableFreeSpace     Indicates the amount of available free space on a drive.
    DriveFormat             Gets the name of the file system, such as NTFS or FAT32.
    DriveType         Gets the drive type.
    IsReady              Gets a value indicating whether a drive is ready.
    Name                 Gets the name of a drive.
    RootDirectory       Gets the root directory of a drive.
    TotalFreeSpace      Gets the total amount of free space available on a drive.
    TotalSize         Gets the total size of storage space on a drive.
    VolumeLabel             Gets or sets the volume label of a drive.
    */
    
    [Microsoft.SqlServer.Server.SqlProcedure]
    public static void spClrLogicalDriveInfo()
    {
        string serverName = Environment.MachineName;
        
        PerformanceCounterCategory pcc = new PerformanceCounterCategory("LogicalDisk", serverName);

        SqlDataRecord record = new SqlDataRecord(
            new SqlMetaData("DriveLetter", SqlDbType.NVarChar, 256),
            new SqlMetaData("TotalSize", SqlDbType.BigInt),
            new SqlMetaData("TotalFreeSpace", SqlDbType.BigInt),
            new SqlMetaData("DriveFormat", SqlDbType.VarChar, 32),
            new SqlMetaData("DriveType", SqlDbType.VarChar, 32),
            new SqlMetaData("VolumeLabel", SqlDbType.VarChar, 20),
            new SqlMetaData("AvailableFreeSpace", SqlDbType.BigInt));

        SqlContext.Pipe.SendResultsStart(record);

        foreach (string instanceName in pcc.GetInstanceNames())
        {
            PerformanceCounter pcPercentFree = new PerformanceCounter("LogicalDisk", "% Free Space", instanceName, serverName);
            PerformanceCounter pcFreeMbytes = new PerformanceCounter("LogicalDisk", "Free Megabytes", instanceName, serverName);

            float percentfree = pcPercentFree.NextValue();

            if (percentfree == 0) { percentfree = 1; };
            
            float TotalFreeSpace = pcFreeMbytes.NextValue();
            float TotalSize = (TotalFreeSpace * 100) / percentfree;

            if (instanceName != "_Total")
            {
                record.SetSqlString(0, instanceName + @"\");                //DriveLetter
                record.SetSqlInt64 (1, Convert.ToInt64(TotalSize));         //TotalSize
                record.SetSqlInt64 (2, Convert.ToInt64(TotalFreeSpace));    //TotalFreeSpace
                record.SetSqlString(3, "");                                 //DriveFormat (not supported by PerfMon)
                record.SetSqlString(4, "");                                 //DriveType (not supported by PerfMon)
                record.SetSqlString(5, "");                                 //VolumeLabel (not supported by PerfMon)
                record.SetSqlInt64 (6, Convert.ToInt64(0));                 //AvailableFreeSpace (not supported by PerfMon)
                SqlContext.Pipe.SendResultsRow(record);  
            }
        }
        SqlContext.Pipe.SendResultsEnd();
    }

    //-------------------------------------------------------------------------------------
    // Performance counters
    //-------------------------------------------------------------------------------------

    // ToDo
};

public partial class UserDefinedFunctions
{

    //-------------------------------------------------------------------------------------
    // Physical disk info (no mountpoint support)
    //-------------------------------------------------------------------------------------
    /*
    AvailableFreeSpace     Indicates the amount of available free space on a drive.
    DriveFormat             Gets the name of the file system, such as NTFS or FAT32.
    DriveType         Gets the drive type.
    IsReady              Gets a value indicating whether a drive is ready.
    Name                 Gets the name of a drive.
    RootDirectory       Gets the root directory of a drive.
    TotalFreeSpace      Gets the total amount of free space available on a drive.
    TotalSize         Gets the total size of storage space on a drive.
    VolumeLabel             Gets or sets the volume label of a drive.
    */

    [SqlFunction(FillRowMethodName = "FillRow",
        TableDefinition = "DriveLetter nvarchar(256)," +
                         "TotalSize bigint null," +
                         "TotalFreeSpace bigint null," +
                         "DriveFormat nvarchar(32) null," +
                         "DriveType nvarchar(32) null," +
                         "VolumeLabel nvarchar(20) null," +
                         "AvailableFreeSpace bigint null"
                         )]

    public static IEnumerable fnClrDriveInfo()
    {
        return System.IO.DriveInfo.GetDrives();
    }
    
    public static void FillRow(Object obj
                             , out string DriveLetter
                             , out SqlInt64 TotalSize
                             , out SqlInt64 TotalFreeSpace
                             , out string DriveFormat
                             , out string DriveType
                             , out string VolumeLabel
                             , out SqlInt64 AvailableFreeSpace
                             )
    {
        DriveInfo drive = (DriveInfo)obj;
        DriveLetter = drive.Name;
        DriveType = drive.DriveType.ToString();

        // Check if the drive is ready (cdrom drives, SD cardreaders etc)
        if (drive.IsReady)
        {
            TotalSize = new SqlInt64(drive.TotalSize) / 1048576;
            TotalFreeSpace = new SqlInt64(drive.TotalFreeSpace) / 1048576;
            DriveFormat = drive.DriveFormat;
            VolumeLabel = drive.VolumeLabel;
            AvailableFreeSpace = drive.AvailableFreeSpace / 1048576;
        }
        else
        {
            TotalSize = new SqlInt64();
            TotalFreeSpace = new SqlInt64();
            DriveFormat = null;
            VolumeLabel = null;
            AvailableFreeSpace = new SqlInt64();
        }
    }

    //-------------------------------------------------------------------------------------
    // Split replacement
    //-------------------------------------------------------------------------------------

    [SqlFunction(Name = "fnClrSplit", 
    FillRowMethodName = "FillSplitRow",
    TableDefinition = "txt nvarchar(10)")]

    public static IEnumerable fnClrSplit(SqlString str, SqlChars delimiter)
    {
        if (delimiter.Length == 0)
            return new string[1] { str.Value };

        return str.Value.Split(delimiter[0]);
    }

    public static void FillSplitRow(object row, out SqlString str)
    {
        str = new SqlString((string)row);
    }

};

----------------------------------------------------------------------------------------------------------------------

And after some testing i rolled it out, hoping my mountpoint info would be returned much faster...

And murphy made damn sure it was'nt....  not even by a long shot :'(

Having learned the hard way never to trust just one measurement, I tested it against 45 SQL instances, and then something caught my eye.....



The more mountpoints there are, the longer it took, and the curve was *no way near linear*.

So this is where i am now....  ,having almost run out of available letters of the alphabet, having no way to stop the grow of the mountpoints or databases, and fast loosing the ability to check to see if the mountpoints are running out of space from within SQL.

So i'm hoping some fellow DBA out there, cursed by as many a mountpoint as i am had cracked a way to get stable performing free space info on mountpoints?

Suggestions anyone?

Sponsored



Replies

Erland Sommarskog on Tue, 24 Feb 2015 22:53:11


Not that I know much about what is going on, but I would convert the code to a stand-alone console program that accepts the mountpoint name gets the information and writes it to the file. And then I would observe how that behaves. I would run it from regular command window, not from SQL Server. And depending on the outcome of that test, I would decide on the next step.

Theo Ekelmans on Wed, 25 Feb 2015 09:10:23


Hi Erland,

I've made a little console app containing the same loop i'm using in the CLR and ran that on my PC and on a server.

Code:

                            
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace ConsoleApplication1
{
    class Program
    {
        public static void Main()
        {
            string serverName = Environment.MachineName;

            DateTime StartDT = DateTime.Now;

            DateTime StartStepDT;
            DateTime EndStepDT;
            double StepRuntimeMs;
            double EnumRuntimeMs;

            StartStepDT = DateTime.Now; 

            PerformanceCounterCategory pcc = new PerformanceCounterCategory("LogicalDisk", serverName);

            EndStepDT = DateTime.Now;
            StepRuntimeMs = (EndStepDT - StartStepDT).TotalMilliseconds;
            Console.WriteLine("Init Runtime:          {0, 8} ms", StepRuntimeMs);

            StartStepDT = DateTime.Now; 
            
            foreach (string instanceName in pcc.GetInstanceNames())
            {
                EndStepDT = DateTime.Now;
                EnumRuntimeMs = (EndStepDT - StartStepDT).TotalMilliseconds;
                
                StartStepDT = DateTime.Now; 

                PerformanceCounter pcPercentFree = new PerformanceCounter("LogicalDisk", "% Free Space", instanceName, serverName);
                PerformanceCounter pcFreeMbytes = new PerformanceCounter("LogicalDisk", "Free Megabytes", instanceName, serverName);

                float percentfree = pcPercentFree.NextValue();

                if (percentfree == 0) { percentfree = 1; };

                float TotalFreeSpace = pcFreeMbytes.NextValue();
                float TotalSize = (TotalFreeSpace * 100) / percentfree;

                if (instanceName != "_Total")
                {
                    Console.WriteLine("Drive {0}", instanceName + @"\");
                    Console.WriteLine("  Total available space:          {0, 8} MB", TotalFreeSpace);
                    Console.WriteLine("  Total size of drive:            {0, 8} MB", TotalSize);


                    EndStepDT = DateTime.Now;
                    StepRuntimeMs = (EndStepDT - StartStepDT).TotalMilliseconds;

                    Console.WriteLine("  Enum Runtime:          {0, 8} ms", EnumRuntimeMs);
                    Console.WriteLine("  Step Runtime:          {0, 8} ms", StepRuntimeMs);
                }
                StartStepDT = DateTime.Now; 
            }

            DateTime EndDT = DateTime.Now;

            double RuntimeMs = (EndDT - StartDT).TotalMilliseconds;

            Console.WriteLine("");
            Console.WriteLine("Runtime:          {0, 8} ms", RuntimeMs);
        }
    }
}



The steps that takes the most time is ofcourse the first Enum step, that has to init everything, 2.7 sec on my laptop and 4.6 sec on the server.  After that, my latop rips throug the for each loop in about 1 ms per loop, but the server takes 300 to 450 ms per loop.

-------------------------------------------------------
laptop

-------------------------------------------------------

Init Runtime:            1,0001 ms
Drive F:\
  Total available space:              3748 MB
  Total size of drive:                3757 MB
  Enum Runtime:          2700,3429 ms
  Step Runtime:            3,0004 ms
Drive HarddiskVolume1\
  Total available space:                70 MB
  Total size of drive:                  99 MB
  Enum Runtime:                 0 ms
  Step Runtime:            0,5001 ms
Drive D:\
  Total available space:            278710 MB
  Total size of drive:              476936 MB
  Enum Runtime:                 0 ms
  Step Runtime:            0,5001 ms
Drive C:\
  Total available space:             75930 MB
  Total size of drive:              244095 MB
  Enum Runtime:                 0 ms
  Step Runtime:            1,5002 ms

Runtime:          2730,8468 ms

--------------------------------------------
Server
--------------------------------------------

Init Runtime:                 0 ms
Drive D:\
  Total available space:             80121 MB
  Total size of drive:              102409 MB
  Enum Runtime:            3931.2 ms
  Step Runtime:             343.2 ms
Drive O:\MSSQL\DATA\DATA9\
  Total available space:             10120 MB
  Total size of drive:               30720 MB
  Enum Runtime:                 0 ms
  Step Runtime:               312 ms

----- snip --------------- removed 120 entries

Drive O:\MSSQL\DATA\DATA6\
  Total available space:              2318 MB
  Total size of drive:               10238 MB
  Enum Runtime:                 0 ms
  Step Runtime:               312 ms
Drive O:\MSSQL\DATA\DATA5\
  Total available space:              8559 MB
  Total size of drive:               10238 MB
  Enum Runtime:                 0 ms
  Step Runtime:               312 ms
Drive R:\
  Total available space:             15763 MB
  Total size of drive:               20479 MB
  Enum Runtime:                 0 ms
  Step Runtime:             296.4 ms

Runtime:           39873.6 ms

--------------------------------------------------------------

So i was wondering Erland, do you know of any other way to get to mountpoint free space faster then this ?

Grtz, T :)


HoroChan on Wed, 25 Feb 2015 09:45:12


Is this something related to SQL Server? how about a stand alone C# project to statistic the mountpoint free space?

Theo Ekelmans on Wed, 25 Feb 2015 09:55:56


Erland suggested that as well

Just posted the results of a standalone console app, your post and that one probably crossed eachother.

Grtz, T :)

Erland Sommarskog on Wed, 25 Feb 2015 22:02:01


So i was wondering Erland, do you know of any other way to get to mountpoint free space faster then this ?

No, I am an SQL Server guy, not a Windows or a .NET guy.

But now you have taken SQL Server out of the equation, it will be easier for you to ask about this in a more appropriate forum. And, no, I don't know what forum that would be, because, as I said, I am an SQL Server guy.

Solomon Rutzky on Mon, 13 Apr 2015 15:55:03


Just FYI, this question was also posted on the SQL Server Central forums: http://www.sqlservercentral.com/Forums/Topic1663173-386-1.aspx

Theo Ekelmans on Mon, 20 Apr 2015 15:34:42


Hi Erland,

Solomon and I came up with a workaround that will fit 99% of all the installations out there :)

http://www.sqlservercentral.com/Forums/Topic1663173-386-1.aspx

Erland Sommarskog on Mon, 20 Apr 2015 21:21:53


Great that you found a solution, even if was a in different corner that you looked in.