Mythtv Storage Groups

Storage Group Notes

On Tue Mar 06, 2007 at 04:53:31PM +1030, Darryl Ross wrote:

> > Firstly, thanks heaps to the devs and testers who have got Storage > > Groups working really well. It's simplified my setup hugely!

Glad it's working well for you. :)

> > Just wondering if there is any capability (or plans) or specify a > > preference to how the storage groups are utilised?

There is limited ability for this via some undocumented settings, but there is no GUI to set them up.

> > At the moment I believe Myth will record to the drive with the most > > amount of free space if there's only a single recording and will spread > > simultaneous the recordings across all available drives to reduce head > > movements.

It's a bit more complicated than that. Myth uses weights to determine what filesystem/directory to record to. The following default values are used:

  SGweightPerRecording  = 10
  SGweightPerPlayback   =  5
  SGweightPerCommFlag   =  5
  SGweightPerTranscode  =  5
  SGweightLocalStarting = -1.99 * SGweightPerRecording
  SGmaxRecOverlapMins   =  3

The drive with the lowest weight is used first. In a tie, the drive with the highest amount of free disk space is used first.

Each Storage Group Directory has its own weight, but Myth is smart enough to know if several directories are on the same shared filesystem. When a weight is applied to a directory that is in use, it is applied to all directories on that filesystem because we are trying to spread the load out across filesystems, not directories on the same filesystem.

The starting weight of all drives is 0. Local filesystems/directories are then offset by SGweightLocalStarting, so by default they are then at -19 because SGweightPerRecording defaults to 10. This makes it so that local drives are preferred over remote drives. A local drive has to have an effective weight of 20 before a remote drive will be used to store a new recording. If you do not have any playback going on, this would mean that you'd have to have 2 recordings going to a local drive before a remote drive would get used. If you have 2 local drives and 1 remote drive, you'd have to have 4 recordings going on locally before a remote drive would get used.

So, if you have less than 4 tuners and 2 local drives, then the default should already give you what you are looking for. If you have more than 4 tuners or want to guarantee that all recordings go to the local drives unless they fill up, you can do so using the undocumented SGweightPerDir setting.

In order to make Myth not use a filesystem/directory, we need to artificially inflate the starting weight for that directory. We can do this by insertting a setting in the database.

The key for the setting is SGweightPerDir:HOSTNAME:DIRECTORY. The hostname is the hostname that sees the directory. So if the directory is actually on the fileserver which is server1 but is mounted via NFS on server2 which is running mythbackend, we'd use server2 here. The directory is the local path on HOSTNAME, so you'd put /mnt/video or whatever you use here for the remotely mounted directory.

  SGweightPerDir:server2:/mnt/video

The value that we put in this setting will be applied as an offset to the initial weight for this directory. You can play it safe by setting this to something large like 99 or 100 or even higher. So in the example here, the actual setting key would be:

You need to run the following SQL to insert that into the settings table:

  INSERT settings
     (value, data, hostname)
  VALUES
     ("SGweightPerDir:server2:/mnt/video", 99, "server2");

So, now when the Storage Groups scheduling code runs, /mnt/video will start out with a weight of 0. Since it is a remote drive, the only offsets that will be initially applied will be the one we specified in the settings table with the SGweightPerDir setting. Your two local directories would each end up starting out at -19 and the remote directory would start out at +99. So unless you have a huge number of recordings and playbacks going on on each of your local drives, the remote directory will never be used unless the locals fill up.

If you run mythbackend with the "-v schedule,file" option, you can see the weights as they are applied and the logs will show you why Myth chose one directory over another when determining where to put the next recording.

-- Chris (who still needs to get around to documenting this more on the wiki or in the HOWTO)