TheFourStagesofNTFSFileGrowth,Part2|AsktheCoreTeam
Server & Tools Blogs > Server & Management Blogs > Ask the Core Team
Sign in
A few years ago I wrote a blog entry entitled, The Four Stages of NTFS File Growth.
This attempted to explain what happens to a file as it gains complexity. Complexity being akin to fragmentation.
If you have not read the above mentioned blog entry, please do so now. This information will not make the
slightest bit of sense unless you read my earlier post. Ill wait.
http://blogs.technet.com/b/askcore/archive/2009/10/16/the
fourstagesofntfsfilegrowth.aspx
Welcome back.
Since its posting, I have answered a number of questions, mostly about the structure called the attribute list. So
today I want to cover this a little more indepth to hopefully address some of these said questions.
In the previous blog entry, I explained how very complex files had the potential of creating an attribute list
shown below.
https://blogs.technet.microsoft.com/askcore/2015/03/12/thefourstagesofntfsfilegrowthpart2/
1/10
10/4/2016
TheFourStagesofNTFSFileGrowth,Part2|AsktheCoreTeam
The base record and all the child records are each 1kb in size. Each child record keeps track of a portion of the
files data stream. The more fragmented the data stream, the more mapping pairs are required to track the
fragments, and thus the more child records will be created. Each child record must be tracked in the attribute list.
Keep in mind that the child records can hold much more than just two mapping pairs. This is just simplified to
keep the diagram from being completely unreadable.
The problem with this is that the attribute list itself. It is NOT a child record, it is created using free space outside
the Master File Table MFT. A files attribute list has a hard limit of how large it can grow. This cannot be changed.
If it were, it would break backwards compatibility with older versions of NTFS that wouldnt know how to deal
with a larger attribute list.
NOTE: The diagram shows the attribute list as being smaller than the 1kb file record. And while it is true that it
starts out that way, the upper limitation of the attribute list is 256kb.
So it is possible to hit a point where a file cannot add on any additional fragments. This is often the case when
the following error messages are encountered.
https://blogs.technet.microsoft.com/askcore/2015/03/12/thefourstagesofntfsfilegrowthpart2/
2/10
10/4/2016
TheFourStagesofNTFSFileGrowth,Part2|AsktheCoreTeam
http://support.microsoft.com/kb/967351/
Installing the hotfix doesnt resolve the issue by itself. What this hotfix does is that it gives us the ability to create
instances of NTFS that use file records that are 4kb in size, rather than the 1kb that NTFS has used for the longest
time.
How is this possible? If we cant change the size of the attribute list, how can we change the size of file records?
The attribute list is a hard coded limitation. Microsoft made the decision, for performance reasons, that we really
should keep a lid on how big the attribute list should grow. On the other hand, file record size is selfdefined. By
default, the size is defined as 1kb, but records could be other sizes, as long as all the records in a volume are the
same size.
This was put to the test when 4kb sector hard drives started to become popular. Since you wouldnt want a file
record to be smaller than a sector, these 4kb sector drives were formatted to utilize a file record size of 4kb. Thats
where the hotfix comes into the picture. In addition to being able to use 4kb file records on 4kb sector hard
drives, an option was added to the FORMAT.EXE command to force it to create an instance of NTFS with 4kb file
records, regardless of sector size.
So why should we care about the size of the file records? Look at the diagram again.
https://blogs.technet.microsoft.com/askcore/2015/03/12/thefourstagesofntfsfilegrowthpart2/
3/10
10/4/2016
TheFourStagesofNTFSFileGrowth,Part2|AsktheCoreTeam
If the records are bigger, they can store more mapping pairs, and thus track more fragments. In theory, a file
could have FOUR TIMES the number of fragments before running into the same issue.
The catch is that the size of file records is set at the time of formatting. So if you have a volume that is running
into this issue, you will need to do the following.
1. Copy off your files
2. Reformat the drive using the switch Format /L
3. Copy the files back
You cant change the size of file records after the fact. It has to be set when formatting. But without an
understanding of just what it is that we are changing.
This solves the problem in the short term. For the long term, other solutions were implemented to prevent
fragmentation past a certain point. In the newer versions of Windows, NTFS will stop fragmenting compressed
and sparse files before the attribute list reaches 100% of its maximum size.
This should put the issue to rest once and for all. However, until everyone gets to Windows 8.1 or Windows
Server 2012 R2, we will still run into this issue from time to time.
For more information about 4kb sector drives, check out my article on Windows IT Pro.
http://windowsitpro.com/windows/promiseadvanced
formatharddrives
https://blogs.technet.microsoft.com/askcore/2015/03/12/thefourstagesofntfsfilegrowthpart2/
4/10
10/4/2016
TheFourStagesofNTFSFileGrowth,Part2|AsktheCoreTeam
Robert Mitchell
Senior Support Escalation Engineer
Microsoft Enterprise Platforms Support
5/10