Difference between revisions of "Very Large Hard Drive"

From hummy.tv Wiki
Jump to: navigation, search
(→‎Formatting the file-systems)
m (→‎Pre-requisites)
 
(17 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
{{Warning|title=Work in progress|This page is a work-in-progress describing the process necessary to install a very large hard drive into a Humax HDR Fox T2.}}
 
{{Warning|title=Work in progress|This page is a work-in-progress describing the process necessary to install a very large hard drive into a Humax HDR Fox T2.}}
  
−
With the release of Custom Firmware 3.10 it is now possible to fit a Humax HDR Fox T2 with a hard drive having more than 2^32 sectors. In practice this means that drives over 2TB in size can now be used.
+
With the release of Custom Firmware 3.10 it is now possible to fit a Humax HDR Fox T2 with a hard drive having more than 2^32 sectors. In practice this means that drives between 2TB and 4TB in size can now be used. Disks larger than 4TB have been found to cause problems with the Humax software reporting low amounts of free space.
  
 
===Drives known to work===
 
===Drives known to work===
  
−
The following very large drives have been tested and are known to work:
+
The following very large drives have been successfully tested:
  
 
{| class=wikitable
 
{| class=wikitable
−
! Vendor || Model || Size || Physical Sector Size || Logical Sector Size || Description
+
! AV? || Vendor || Description || Model || Size || Cache || RPM || Physical Sector || Logical Sector
 
|-
 
|-
−
| Seagate || ST4000VM000 || 4TB || 4096 || 512 || Video 3.5 HDD SATA
+
| * || Seagate || Video 3.5" HDD SATA || ST4000VM000 || 4TB || 64MB || 5900 || 4096 || 512
 +
|-
 +
|  || Seagate || NAS 3.5" HDD SATA || ST4000VN000 || 4TB || 64MB || 5900 || 4096 || 512
 +
|-
 +
|  || Western Digital || Green.. TBC || || 4TB
 
|}
 
|}
 +
 +
{{Note|'''Always choose a specific Audio/Visual (AV) or Digital Video Recorder (DVR) drive if possible.'''
 +
 +
* Avoid drives which are designed for RAID use (e.g. WD Red and Purple drives) as they will have TLER (an error recovery option) enabled which can cause issues if there are problems with reading or writing to a sector. AV drives usually have this parameter disabled so that a failed write is just ignored and recording moves on. It can also cause complications when dealing with any problem sectors in the future as it causes some issues to be hidden from the host (the Humax in this scenario).
 +
* WD Purple drives have a surprisingly low workload rating of 60TB/year and should be avoided.
 +
* With any non-AV Western Digital disk, watch the disk stats to check if the load/unload count grows quickly which could indicate that automatic head parking after 8 seconds is enabled. This can (and should) be disabled using ''wdidle3''[http://www.storagereview.com/how_to_stop_excessive_load_cycles_on_the_western_digital_2tb_caviar_green_wd20ears_with_wdidle3]. }}
  
 
===Pre-requisites===
 
===Pre-requisites===
  
−
Using a very large drive requires Custom Firmware 3.10 or above. The stock Humax firmware cannot read these drives at all (it will see them but consider them to have a zero size).
+
Using a very large drive '''''requires''''' Custom Firmware 3.10 or above. The stock Humax firmware cannot read these drives at all (it will see them but consider them to have a zero size).
  
 
===Partitions tables===
 
===Partitions tables===
  
−
With these large drives, it is necessary to use a partitioning scheme called GUID Partition Table (GPT). When using a GPT, a legacy MBR containing a partition table is also created on the drive. This MBR generally contains a single partition of type ''EFI GPT'' (code 0xee) encompassing the entire drive, well as much of the drive as can be represented in an MBR partition - 2^32 sectors. This is known as a ''protective MBR'' since its main purposes is to prevent non-GPT-aware systems and tools from viewing the drive as empty or available. In the case of the Humax, it is necessary to create three partitions on the drive so that the Humax software sees these and considers the drive to be properly formatted.
+
With these large drives, it is necessary to use a partitioning scheme called GUID Partition Table (GPT). When using a GPT, a legacy MBR containing a partition table is also created on the drive. This MBR generally contains a single partition of type ''EFI GPT'' (code 0xee) encompassing the entire drive (or as much of the drive as can be represented in an MBR partition - 2^32 sectors). This is known as a ''protective MBR'' since its main purposes is to prevent non-GPT-aware systems and tools from viewing the drive as empty or available. In the case of the Humax, it is necessary to create three partitions on the drive so that the Humax software sees these and considers the drive to be properly formatted.
  
−
===Creating the GPT===
+
===Creating the GPT and formatting the disk===
  
−
Once the large disk is installed the ''gdisk'' command can be used to create the GPT. This command will also create a new protective MBR.
+
Restart the Humax into Maintenance mode then use the telnet menu to do the work for you. Just choose the ''gptf'' option and confirm that you really want to reformat the disk.
  
−
  humax# '''gdisk /dev/sda'''
+
  Menu version 1.20
−
  GPT fdisk (gdisk) version 1.0.1
+
Enter system PIN: ****
 +
 +
      /---------------------------------------------\
 +
      |  M A I N T E N A N C E  M O D E  M E N U  |
 +
      \---------------------------------------------/
 +
   
 +
  [ Humax HDR-Fox T2 (gpttest) 1.03.12/3.10 ]
 
   
 
   
−
Partition table scan:
+
  Note: Some areas of the hard disk are not mounted.
−
  MBR: hybrid
+
        Not all functions will be available.
−
  BSD: not present
+
        (normal if you have just run a disk check)
−
  APM: not present
 
−
  GPT: not present
 
 
   
 
   
−
  ***************************************************************
+
    1 - Check and repair hard disk (fix-disk).
−
  Found invalid GPT and valid MBR; converting MBR to GPT format
+
    2 - Run short hard-disk self test.
−
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
+
    3 - Run long hard-disk self test.
−
typing 'q' if you don't want to convert your MBR partitions
+
    4 - Check self-test progress.
−
to GPT format!
+
  gptf - Re-format disk using GPT scheme.
−
  ***************************************************************
+
  epg - Clear persistent EPG data.
−
 
+
  dlna - Reset DLNA server database.
−
Delete all partitions:
+
    x - Leave maintenance mode (Humax will restart).
−
 
+
  diag - Run a diagnostic.
−
  Command (? for help): '''o'''
+
  cli - System command line (advanced users).
−
  This option deletes all partitions and creates a new protective MBR.
+
−
Proceed? (Y/N): '''y'''
+
Please select option: gptf
−
 
+
−
Create the three required partitions:
+
This process will format the internal hard disk using the GUID partitioning
−
 
+
scheme (GPT) which supports hard disks over 2TB in size.
−
  Command (? for help): '''n'''
+
−
  Partition number (1-128, default 1): '''1'''
+
  ****************************************************************
−
First sector (34-7814037134, default = 2048) or {+-}size{KMGTP}:
+
  *** All data on the hard drive will be erased if you proceed ***
−
Last sector (2048-7814037134, default = 7814037134) or {+-}size{KMGTP}: '''+1G'''
+
  ****************************************************************
−
Current type is 'Linux filesystem'
+
   
−
Hex code or GUID (L to show codes, Enter = 8300):
+
  Are you sure you wish to format and erase the disk? [Y/N] y
−
Changed type of partition to 'Linux filesystem'
+
  Are you sure you wish to do this? [Y/N] y
 +
  Are you sure you wish to COMPLETELY ERASE this hard disk? [Y/N] y
 
   
 
   
−
  Command (? for help): '''n'''
+
  **********************************************************************
−
  Partition number (2-128, default 2): '''2'''
+
  * Unmounting partitions...
−
  First sector (34-7814037134, default = 2099200) or {+-}size{KMGTP}:
+
  **********************************************************************
−
Last sector (2099200-7814037134, default = 7814037134) or {+-}size{KMGTP}: '''-10G'''
 
−
Current type is 'Linux filesystem'
 
−
Hex code or GUID (L to show codes, Enter = 8300):
 
−
Changed type of partition to 'Linux filesystem'
 
 
   
 
   
−
  Command (? for help): '''n'''
+
  **********************************************************************
−
  Partition number (3-128, default 3): '''3'''
+
  * Clearing all existing partition tables...
−
First sector (34-7814037134, default = 7793065984) or {+-}size{KMGTP}:
+
  **********************************************************************
−
Last sector (7793065984-7814037134, default = 7814037134) or {+-}size{KMGTP}:
 
−
Current type is 'Linux filesystem'
 
−
Hex code or GUID (L to show codes, Enter = 8300):
 
−
Changed type of partition to 'Linux filesystem'
 
−
 
 
−
Which results in the following table (for a 4TB drive):
 
−
 
 
−
Command (? for help): p
 
−
Disk /dev/sda: 7814037168 sectors, 3.6 TiB
 
−
Logical sector size: 512 bytes
 
−
Disk identifier (GUID): C4FC1347-AF56-4BCA-B918-623B08454E30
 
−
  Partition table holds up to 128 entries
 
−
First usable sector is 34, last usable sector is 7814037134
 
−
Partitions will be aligned on 2048-sector boundaries
 
−
Total free space is 2383 sectors (1.2 MiB)
 
 
   
 
   
−
  Number  Start (sector)    End (sector)  Size      Code  Name
+
  Creating new GPT entries.
−
    1            2048        2099199  1024.0 MiB  8300  Linux filesystem
+
  GPT data structures destroyed! You may now partition the disk using fdisk or
−
    2        2099200      7793065614  3.6 TiB    8300 Linux filesystem
+
  other utilities.
−
    3      7793065984      7814037134  10.0 GiB    8300  Linux filesystem
 
−
 
 
−
Now write out the partition table:
 
−
 
 
−
  Command (? for help): w
 
 
   
 
   
−
  Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
+
  **********************************************************************
−
  PARTITIONS!!
+
* Creating GUID partition table (GPT)...
 +
  **********************************************************************
 
   
 
   
−
  Do you want to proceed? (Y/N): y
+
  Creating new GPT entries.
−
OK; writing new GUID partition table (GPT) to /dev/sda.
 
 
  The operation has completed successfully.
 
  The operation has completed successfully.
−
 
−
===Creating the MBR Partition Table===
 
−
 
−
First remove the protective partition that will have been created by the ''gdisk'' utility in the previous step:
 
−
 
−
humax# '''fdisk /dev/sda'''
 
−
fdisk: device has more than 2^32 sectors, can't use all of them
 
 
   
 
   
−
  The number of cylinders for this disk is set to 266305.
+
  **********************************************************************
−
  There is nothing wrong with that, but this is larger than 1024,
+
  * Creating MBR partition table...
−
  and could in certain setups cause problems with:
+
  **********************************************************************
−
  1) software that runs at boot time (e.g., old versions of LILO)
+
   
−
  2) booting and partitioning software from other OSs
+
  >>> /dev/sda2: /dev/sda3: /dev/sda4:
−
    (e.g., DOS FDISK, OS/2 FDISK)
 
 
   
 
   
−
  Command (m for help): '''p'''
+
  **********************************************************************
 +
* Final MBR table...
 +
**********************************************************************
 
   
 
   
−
  Disk /dev/sda: 2199.0 GB, 2199023255040 bytes
+
  Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
−
256 heads, 63 sectors/track, 266305 cylinders
+
  Units: sectors of 1 * 512 = 512 bytes
−
  Units = cylinders of 16128 * 512 = 8257536 bytes
+
Sector size (logical/physical): 512 bytes / 512 bytes
 +
I/O size (minimum/optimal): 512 bytes / 512 bytes
 +
Disklabel type: dos
 +
Disk identifier: 0x0c82049c
 
   
 
   
−
     Device Boot     Start         End     Blocks Id System
+
Device     Boot   Start     End Sectors Size Id Type
−
  /dev/sda1               1     266306  2147483647+ ee EFI GPT
+
  /dev/sda1             1 2097151 2097151 1024M ee GPT
 +
/dev/sda2      2097152 4194303 2097152    1G 83 Linux
 +
/dev/sda3      4194304 6291455 2097152    1G 83 Linux
 
   
 
   
−
  Command (m for help): '''d'''
+
  **********************************************************************
−
  Selected partition 1
+
  * Final GPT table...
−
 
+
  **********************************************************************
−
Then create the three dummy partitions for the Humax software to see:
 
−
 
 
−
Command (m for help): '''n'''
 
−
Command action
 
−
    e  extended
 
−
    p  primary partition (1-4)
 
−
'''p'''
 
−
Partition number (1-4): '''1'''
 
−
First cylinder (1-266305, default 1): Using default value 1
 
−
  Last cylinder or +size or +sizeM or +sizeK (1-266305, default 266305): '''+1G''' 
 
 
   
 
   
−
  Command (m for help): '''n'''
+
  Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
−
  Command action
+
  Units: sectors of 1 * 512 = 512 bytes
−
    e  extended
+
Sector size (logical/physical): 512 bytes / 512 bytes
−
    p  primary partition (1-4)
+
  I/O size (minimum/optimal): 512 bytes / 512 bytes
−
  '''p'''
+
  Disklabel type: gpt
−
Partition number (1-4): '''2'''
+
  Disk identifier: 372B3BA4-7C50-4D29-8551-4A94E7826C05
−
  First cylinder (123-266305, default 123): Using default value 123
 
−
  Last cylinder or +size or +sizeM or +sizeK (123-266305, default 266305): '''+1G'''
 
 
   
 
   
−
  Command (m for help): '''n'''
+
  Device        Start      End  Sectors  Size Type
−
  Command action
+
  /dev/sda1      2048  2099199   2097152     1G Linux filesystem
−
    e   extended
+
  /dev/sda2    2099200 955801614 953702415 454.8G Linux filesystem
−
     p  primary partition (1-4)
+
  /dev/sda3  955803648 976773134 20969487    10G Linux filesystem
−
'''p'''
 
−
Partition number (1-4): '''3'''
 
−
First cylinder (245-266305, default 245): Using default value 245
 
−
Last cylinder or +size or +sizeM or +sizeK (245-266305, default 266305): '''+1G'''
 
−
 
 
−
Resulting in the following partition table:
 
−
 
 
−
  Command (m for help): '''p'''
 
−
Disk /dev/sda: 2199.0 GB, 2199023255040 bytes
 
−
  256 heads, 63 sectors/track, 266305 cylinders
 
−
  Units = cylinders of 16128 * 512 = 8257536 bytes
 
 
   
 
   
−
    Device Boot      Start        End      Blocks Id System
+
  **********************************************************************
−
  /dev/sda1              1        122      983776+ 83 Linux
+
  * Formatting partition 1...
−
/dev/sda2            123        244      983808  83 Linux
+
  **********************************************************************
−
/dev/sda3            245        366      983808  83 Linux
 
−
 
 
−
Change the type of the first partiton to EFI GPT:
 
−
 
 
−
Command (m for help): '''t'''
 
−
Partition number (1-4): '''1'''
 
−
Hex code (type L to list codes): '''ee'''
 
−
Changed system type of partition 1 to ee (EFI GPT)
 
−
 
 
−
Move the first sector of the first partition to 1:
 
−
 
 
−
Command (m for help): '''u'''
 
−
  Changing display/entry units to sectors
 
 
   
 
   
−
  Command (m for help): '''x'''
+
  mke2fs 1.42.10 (18-May-2014)
 +
Creating filesystem with 262144 4k blocks and 65536 inodes
 +
Filesystem UUID: a442c849-f26e-4dd0-b1fe-cef4cf8416b5
 +
Superblock backups stored on blocks:
 +
        32768, 98304, 163840, 229376
 
   
 
   
−
  Expert command (m for help): '''b'''
+
  Allocating group tables: done
−
  Partition number (1-4): '''1'''
+
  Writing inode tables: done
−
  New beginning of data (0-1967615, default 63): '''1'''
+
  Creating journal (8192 blocks): done
−
  Recalculate C/H/S values? (Y/N): '''n'''
+
  Writing superblocks and filesystem accounting information: done
 
   
 
   
−
  Expert command (m for help): '''r'''
+
  .... continues to format partitions 2 & 3 ...
−
 
 
−
The final MBR partition table looks like:
 
−
 
 
−
Command (m for help): '''p'''
 
−
 
 
−
Disk /dev/sda: 2199.0 GB, 2199023255040 bytes
 
−
256 heads, 63 sectors/track, 266305 cylinders, total 4294967295 sectors
 
−
Units = sectors of 1 * 512 = 512 bytes
 
−
 
 
−
    Device Boot      Start        End      Blocks  Id System
 
−
/dev/sda1              1    1967615      983807+ ee EFI GPT
 
−
/dev/sda2        1967616    3935231      983808  83 Linux
 
−
/dev/sda3        3935232    5902847      983808  83 Linux
 
−
 
 
−
Write it to disk:
 
−
 
 
−
Command (m for help): '''w'''
 
−
The partition table has been altered.
 
−
Calling ioctl() to re-read partition table
 
−
 
 
−
Now restart the Humax using the remote control.
 
−
 
 
−
===Formatting the file-systems===
 
−
 
 
−
Once the Humax comes back, the filesystems must be formatted from the command line:
 
−
 
 
−
humax# '''mkfs.ext3 -m 0 -O sparse_super /dev/sda1'''
 
−
humax# '''mkfs.ext3 -m 0 -O sparse_super -T largefile /dev/sda2'''
 
−
humax# '''mkfs.ext3 -m 0 -O sparse_super /dev/sda3'''
 

Latest revision as of 13:19, 17 February 2019


Warning

Work in progress

This page is a work-in-progress describing the process necessary to install a very large hard drive into a Humax HDR Fox T2.


With the release of Custom Firmware 3.10 it is now possible to fit a Humax HDR Fox T2 with a hard drive having more than 2^32 sectors. In practice this means that drives between 2TB and 4TB in size can now be used. Disks larger than 4TB have been found to cause problems with the Humax software reporting low amounts of free space.

Drives known to work

The following very large drives have been successfully tested:

AV? Vendor Description Model Size Cache RPM Physical Sector Logical Sector
* Seagate Video 3.5" HDD SATA ST4000VM000 4TB 64MB 5900 4096 512
Seagate NAS 3.5" HDD SATA ST4000VN000 4TB 64MB 5900 4096 512
Western Digital Green.. TBC 4TB

Note Note: Always choose a specific Audio/Visual (AV) or Digital Video Recorder (DVR) drive if possible.

  • Avoid drives which are designed for RAID use (e.g. WD Red and Purple drives) as they will have TLER (an error recovery option) enabled which can cause issues if there are problems with reading or writing to a sector. AV drives usually have this parameter disabled so that a failed write is just ignored and recording moves on. It can also cause complications when dealing with any problem sectors in the future as it causes some issues to be hidden from the host (the Humax in this scenario).
  • WD Purple drives have a surprisingly low workload rating of 60TB/year and should be avoided.
  • With any non-AV Western Digital disk, watch the disk stats to check if the load/unload count grows quickly which could indicate that automatic head parking after 8 seconds is enabled. This can (and should) be disabled using wdidle3[1].

Pre-requisites

Using a very large drive requires Custom Firmware 3.10 or above. The stock Humax firmware cannot read these drives at all (it will see them but consider them to have a zero size).

Partitions tables

With these large drives, it is necessary to use a partitioning scheme called GUID Partition Table (GPT). When using a GPT, a legacy MBR containing a partition table is also created on the drive. This MBR generally contains a single partition of type EFI GPT (code 0xee) encompassing the entire drive (or as much of the drive as can be represented in an MBR partition - 2^32 sectors). This is known as a protective MBR since its main purposes is to prevent non-GPT-aware systems and tools from viewing the drive as empty or available. In the case of the Humax, it is necessary to create three partitions on the drive so that the Humax software sees these and considers the drive to be properly formatted.

Creating the GPT and formatting the disk

Restart the Humax into Maintenance mode then use the telnet menu to do the work for you. Just choose the gptf option and confirm that you really want to reformat the disk. 

Menu version 1.20
Enter system PIN: ****

      /---------------------------------------------\
      |  M A I N T E N A N C E   M O D E   M E N U  |
      \---------------------------------------------/

  [ Humax HDR-Fox T2 (gpttest) 1.03.12/3.10 ]

  Note: Some areas of the hard disk are not mounted.
        Not all functions will be available.
        (normal if you have just run a disk check)

    1 - Check and repair hard disk (fix-disk).
    2 - Run short hard-disk self test.
    3 - Run long hard-disk self test.
    4 - Check self-test progress.
 gptf - Re-format disk using GPT scheme.
  epg - Clear persistent EPG data.
 dlna - Reset DLNA server database.
    x - Leave maintenance mode (Humax will restart).
 diag - Run a diagnostic.
  cli - System command line (advanced users).

Please select option: gptf

This process will format the internal hard disk using the GUID partitioning
scheme (GPT) which supports hard disks over 2TB in size.

****************************************************************
*** All data on the hard drive will be erased if you proceed ***
****************************************************************

Are you sure you wish to format and erase the disk? [Y/N] y
Are you sure you wish to do this? [Y/N] y
Are you sure you wish to COMPLETELY ERASE this hard disk? [Y/N] y

**********************************************************************
* Unmounting partitions...
**********************************************************************

**********************************************************************
* Clearing all existing partition tables...
**********************************************************************

Creating new GPT entries.
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.

**********************************************************************
* Creating GUID partition table (GPT)...
**********************************************************************

Creating new GPT entries.
The operation has completed successfully.

**********************************************************************
* Creating MBR partition table...
**********************************************************************

>>> /dev/sda2: /dev/sda3: /dev/sda4:

**********************************************************************
* Final MBR table...
**********************************************************************

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0c82049c

Device     Boot   Start     End Sectors  Size Id Type
/dev/sda1             1 2097151 2097151 1024M ee GPT
/dev/sda2       2097152 4194303 2097152    1G 83 Linux
/dev/sda3       4194304 6291455 2097152    1G 83 Linux

**********************************************************************
* Final GPT table...
**********************************************************************

Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 372B3BA4-7C50-4D29-8551-4A94E7826C05

Device         Start       End   Sectors   Size Type
/dev/sda1       2048   2099199   2097152     1G Linux filesystem
/dev/sda2    2099200 955801614 953702415 454.8G Linux filesystem
/dev/sda3  955803648 976773134  20969487    10G Linux filesystem

**********************************************************************
* Formatting partition 1...
**********************************************************************

mke2fs 1.42.10 (18-May-2014)
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: a442c849-f26e-4dd0-b1fe-cef4cf8416b5
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376

Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

.... continues to format partitions 2 & 3 ...
Retrieved from "https://wiki.hummy.tv/w/index.php?title=Very_Large_Hard_Drive&oldid=5173"