Saturday 5 April 2014

super block - filesystem metadata - sort and hard link

Following command displays primary and backup superblock location on /dev/sda3:
# dumpe2fs /dev/hda3 | grep -i superblock


But what is in a File system?

Again file system divided into two categories:

    User data - stores actual data contained in files
    Metadata - stores file system structural information such as superblock, inodes, directories



Filesystem Failures:

Most of time fsck (front end to ext2/ext3 utility) can fix the problem, first simply run e2fsck - to check a Linux ext2/ext3 file system (assuming /home [/dev/sda3 partition] filesystem for demo purpose), first unmount /dev/sda3 then type following command :
# e2fsck -f /dev/sda3


Where,

    -f : Force checking even if the file system seems clean.
   
    Please note that If the superblock is not found, e2fsck will terminate with a fatal error. However Linux maintains multiple redundant copies of the superblock in every file system, so you can use -b {alternative-superblock} option to get rid of this problem. The location of the backup superblock is dependent on the filesystem's blocksize:

    For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
    For filesystems with 2k blocksizes, at block 16384
    For 4k blocksizes, at block 32768.

Tip you can also try any one of the following command(s) to determine alternative-superblock locations:
# mke2fs -n /dev/sda3
OR
# dumpe2fs /dev/sda3|grep -i superblock


To repair file system by alternative-superblock use command as follows:
# e2fsck -f -b 8193 /dev/sda3

However it is highly recommended that you make backup before you run fsck command on system, use dd command to create a backup (provided that you have spare space under /disk2)
# dd if=/dev/sda2 of=/disk2/backup-sda2.img



# Understanding UNIX / Linux filesystem Inodes


The inode (index node) is a fundamental concept in the Linux and UNIX filesystem. Each object in the filesystem is represented by an inode. But what are the objects? Let us try to understand it in simple words. Each and every file under Linux (and UNIX) has following attributes:

########## Inode stores ###############
=> File type (executable, block special etc)
=> Permissions (read, write etc)
=> Owner
=> Group
=> File Size
=> File access, change and modification time (remember UNIX or Linux never stores file creation time, this is favorite question asked in UNIX/Linux sys admin job interview)
=> File deletion time
=> Number of links (soft/hard)
=> Extended attribute such as append only or no one can delete file including root user (immutability)
=> Access Control List (ACLs)

###########################################

All the above information stored in an inode. In short the inode identifies the file and its attributes (as above) . Each inode is identified by a unique inode number within the file system. Inode is also know as index number.


Inode application

Many commands used by system administrators in UNIX / Linux operating systems often give inode numbers to designate a file. Let us see he practical application of inode number. Type the following commands:
$ cd /tmp
$ touch \"la*
$ ls -l

Now try to remove file "la*

You can't, to remove files having created with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. You have to use inode number to remove file.







# How to: Linux / UNIX Delete or Remove Files With Inode Number

An inode identifies the file and its attributes such as file size, owner, and so on. A unique inode number within the file system identifies each inode. But, why to delete file by an inode number? Sure, you can use rm command to delete file. Sometime accidentally you creates filename with control characters or characters which are unable to be input on a keyboard or special character such as ?, * ^ etc. Removing such special character filenames can be problem. Use following method to delete a file with strange characters in its name:

Please note that the procedure outlined below works with Solaris, FreeBSD, Linux, or any other Unixish oses out there:
Find out file inode

First find out file inode number with any one of the following command:

stat {file-name}

OR

ls -il {file-name}
Use find command to remove file:

Use find command as follows to find and remove a file:

find . -inum [inode-number] -exec rm -i {} \;

When prompted for confirmation, press Y to confirm removal of the file.




###
Understanding UNIX / Linux symbolic (soft) and hard links

Hard link vs. Soft link in Linux or UNIX

    Hard links cannot link directories.
    Cannot cross file system boundaries.

Soft or symbolic links are just like hard links. It allows to associate multiple filenames with a single file. However, symbolic links allows:

    To create links between directories.
    Can cross file system boundaries.

These links behave differently when the source of the link is moved or removed.

    Symbolic links are not updated.
    Hard links always refer to the source, even if moved or removed.
   
   

# Why isn’t it possible to create hard links across file system boundaries?

A single inode number use to represent file in each file system. All hard links based upon inode number.

So linking across file system will lead into confusing references for UNIX or Linux. For example, consider following scenario

* File system: /home
* Directory: /home/vivek
* Hard link: /home/vivek/file2
* Original file: /home/vivek/file1

Now you create a hard link as follows:
$ touch file1
$ ln file1 file2
$ ls -l

Output:

-rw-r--r--  2 vivek vivek    0 2006-01-30 13:28 file1
-rw-r--r--  2 vivek vivek    0 2006-01-30 13:28 file2

Now just see inode of both file1 and file2:
$ ls -i file1
782263
$ ls -i file2
782263

As you can see inode number is same for hard link file called file2 in inode table under /home file system. Now if you try to create a hard link for /tmp file system it will lead to confusing references for UNIX or Linux file system. Is that a link no. 782263 in the /home or /tmp file system? To avoid this problem UNIX or Linux does not allow creating hard links across file system boundaries.



######## What is a Superblock, Inode, Dentry and a File?



First and foremost, and I realize that it was not one of the terms from your question, you must understand metadata. Succinctly, and stolen from Wikipedia, metadata is data about data. That is to say that metadata contains information about a piece of data. For example, if I own a car then I have a set of information about the car but which is not part of the car itself. Information such as the registration number, make, model, year of manufacture, insurance information, and so on. All of that information is collectively referred to as the metadata. In Linux and UNIX file systems metadata exists at multiple levels of organization as you will see.

The superblock is essentially file system metadata and defines the file system type, size, status, and information about other metadata structures (metadata of metadata). The superblock is very critical to the file system and therefore is stored in multiple redundant copies for each file system. The superblock is a very "high level" metadata structure for the file system. For example, if the superblock of a partition, /var, becomes corrupt then the file system in question (/var) cannot be mounted by the operating system. Commonly in this event fsck is run and will automatically select an alternate, backup copy of the superblock and attempt to recover the file system. The backup copies themselves are stored in block groups spread through the file system with the first stored at a 1 block offset from the start of the partition. This is important in the event that a manual recovery is necessary. You may view information about superblock backups with the command dumpe2fs /dev/foo | grep -i superblock which is useful in the event of a manual recovery attempt. Let us suppose that the dumpe2fs command outputs the line Backup superblock at 163840, Group descriptors at 163841-163841. We can use this information, and additional knowledge about the file system structure, to attempt to use this superblock backup: /sbin/fsck.ext3 -b 163840 -B 1024 /dev/foo. Please note that I have assumed a block size of 1024 bytes for this example.

An inode exists in, or on, a file system and represents metadata about a file. For clarity, all objects in a Linux or UNIX system are files; actual files, directories, devices, and so on. Please note that, among the metadata contained in an inode, there is no file name as humans think of it, this will be important later. An inode contains essentially information about ownership (user, group), access mode (read, write, execute permissions) and file type.

A dentry is the glue that holds inodes and files together by relating inode numbers to file names. Dentries also play a role in directory caching which, ideally, keeps the most frequently used files on-hand for faster access. File system traversal is another aspect of the dentry as it maintains a relationship between directories and their files.

A file, in addition to being what humans typically think of when presented with the word, is really just a block of logically related arbitrary data. Comparatively very dull considering all of the work done (above) to keep track of them.

I fully realize that a few sentences do not provide a full explanation of any of these concepts so please feel free to ask for additional details when and where necessary.





File

A file just means a bunch of bytes arranged in a certain order. It's what normal people call the contents of a file. When Linux opens a file, it also creates a file object, which holds data about where the file is stored and what processes are using it. The file object (but not the file data itself) is thrown away when the file is closed.

Inode

An inode (short for "index node") is a bunch of attributes about a file that Linux stores. There is one inode for each file (though with some filesystems, Linux has to create its own inodes because the information is spread around the filesystem). The inode stores information like who owns the file, how big the file is, and who is allowed to open the file. Each inode also contains a number unique to the filesystem partition; it's like a serial number for the file described by that inode.

Dentry

A dentry (short for "directory entry") is what the Linux kernel uses to keep track of the hierarchy of files in directories. Each dentry maps an inode number to a file name and a parent directory.

Superblock

The superblock is a unique data structure in a filesystem (though multiple copies exist to guard against corruption). The superblock holds metadata about the filesystem, like which inode is the top-level directory and the type of filesystem used.



In simplicity, dentry and inode are the same thing, an abstraction of files and directories. The differences between dentry and inode are that dentry is used to facilitate directory-specific operations, inode is just a collection of metadata about files and directories. Superblock is the abstraction of filesystem.


###########

lsof:

The manpage of lsof on my Debian system says “When +L is followed by a number, only files having a link count less than that number will be listed.”

i.e: lsof +L1 [ will display all the files that is having 0 ref count => deleted files. ]



########


The attributes as handled by lsattr/chattr on Linux and some of which can be stored by quite a few file systems (ext2/3/4, reiserfs, JFS, OCFS2, btrfs, XFS, nilfs2, hfsplus...) and even queried over CIFS/SMB (when with POSIX extensions) are flags. Just bits than can be turned on or off to disable or enable an attribute (like immutable or archive...). How they are stored is file system specific, but generally as a 16/32/64 bit record in the inode.

The full list of flags is found on Linux native filesystems (ext2/3/4, btrfs...) though not all of the flags apply to all of FS, and for other non-native FS, Linux tries to map them to equivalent features in the corresponding file system. For instance the simmutable flag as stored by OSX on HFS+ file systems is mapped to the corresponding immutable flag in Linux chattr. What flag is supported by what file system is hardly documented at all. Often, reading the kernel source code is the only option.

Extended attributes on the other hand, as set with setfattr or attr on Linux store more than flags. They are attached to a file as well, and are key/value pairs that can be (both key and value) arbitrary arrays of bytes (though with limitation of size on some file systems).

The key can be for instance: system.posix_acl_access or user.rsync.%stat. The system namespace is reserved for the system (you wouldn't change the POSIX ACLs with setfattr, but more with setfacl, POSIX ACLs just happen to be stored as extended attributes at least on some file systems), while the user namespace can be used by applications (here rsync uses it for its --fake-super option, to store information about ownership or permissions when you're not superuser).

Again, how they are stored is filesystem specific. See WikiPedia for more information.


##########
http://unix.stackexchange.com/questions/117093/find-where-inodes-are-being-used


############

Try this with GNU find:

find /start/dir -L -samefile /file/to/check -exec ls -li {} \;

Example output:

1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x1
1234704 -rw-r--r-- 2 user1 user1 1134 2009-09-11 11:12 ./x2
1234983 lrwxrwxrwx 1 user1 user1 2 2009-10-31 16:56 ./testx -> x1
2345059 lrwxrwxrwx 1 user2 user2 2 2010-01-03 16:17 ./x3 -> x1




No comments:

Post a Comment