And I also found the following solution will also apply to any VMWare virtual machine which running Linux OS with LVM.
Expand the disk
Turn off the virtual machine that you wish to extend disk space.
You can expand the virtual disk size by entering ESX server hidden console mode or using VMware Infrastructure Client to remote manage your ESX Server. I will introduce both way here.
- Method One:
Since the GUI mode is easy. Click Your VMWare Guest Machine in your remote management console from VMware Infrastructure Client. Then click "Edit Virtual Machine Settings" from "Getting Started" which is in the Right Panel. Select Hardware -> Hard Disk 1 -> Capacity - New Size (at Right Side), and then expand the Capacity to whatever size that you need. In our case, I expanded from 100GB to 200GB. Then press "OK". - Method Two:
For ESXi Server, using the following command to enter ESXi hidden console mode after press "Alt"+"F1":unsupported
Then press "Enter", now enter your ESXi server root password. You should be able to successfully log into the hidden ESXi Server console.
For ESX Server, If you press Alt-F1, you can get access to the Linux login prompt of the ESX service console and login to a command prompt.
Now enter the following command to expand the VM Disk:
vmkfstools -X 200G "Your VMDK file here"To extend an existing Virtual Disk to 200GB.
Note:
If you are running VMware workstation product, then you can expand the virtual disk to expand the virtual disk on your Windows machine by running this command:
vmware-vdiskmanager -x 200G "My harddisk.vmdk"After above steps, you finished preparing the extra virtual disk space. Now we can start your VMware virtual machine and open a terminal session to continue expanding your LVM.
Issue the df -k command which shows us that the logical volume is at 100% usage. We need to create a partition on /dev/sda. Type ls -al /dev/sda* which lists the existing partitions. Our new partition will be /dev/sda3.
# ls -al /dev/sda*
brw-r----- 1 root disk 8, 0 Apr 2 19:26 /dev/sda
brw-r----- 1 root disk 8, 1 Apr 2 19:26 /dev/sda1
brw-r----- 1 root disk 8, 2 Apr 2 19:26 /dev/sda2
Type fdisk /dev/sda then type n for new partition. Enter p and 3 for the partition number (in this instance, obviously enter the partition number that matches your environment.) Also accept the default First and Last cylinders. Finally type w to write table to disk and exit.
If prompted to reboot then do so to ensure that the new partition table is written. After a restart, type ls -al /dev/sda* to verify whether the new partition was created successfully.
# ls -al /dev/sda*
brw-r----- 1 root disk 8, 0 Apr 2 19:26 /dev/sda
brw-r----- 1 root disk 8, 1 Apr 2 19:26 /dev/sda1
brw-r----- 1 root disk 8, 2 Apr 2 19:26 /dev/sda2
brw-r----- 1 root disk 8, 3 Apr 2 19:26 /dev/sda3
After verification of new partition, we need to create a physical volume and add it to the volume group.
Type pvcreate /dev/sda3
[root@rhel5 ~]# pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created
Type vgextend VolGroup00 /dev/sda3
# vgextend VolGroup00 /dev/sda3
Volume group "VolGroup00" successfully extended
We need to extend the logical volume. Type vgdisplay:
# vgdisplay
--- Volume group ---
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 99.84 GB
PE Size 32.00 MB
Total PE 3345
Alloc PE / Size 2868 / 99.88 GB
Free PE / Size 2870 / 99.97 GB
VG UUID MkluWy-e5PA-QTGN-fF7k-ZxO3-6bC7-qxfCii
Which shows that there is 99.97GB free that we can add to the volume group. To extend the volume type lvextend -L +99.97G /dev/VolGroup00/LogVol00. If you got an error message that stated that there was "Insufficient free space: 2871 extends needed but only 2870 evailable". After a quick google: Insufficient Free Extents for a Logical Volume. It seems that we needed to select a lightly smaller size. So we changed to 99GB rather than 99.97GB which solved this problem.
# lvextend -L+99G /dev/VolGroup00/LogVol00
Rounding up size to full physical extent 99 GB
Extending logical volume LogVol00 to 198 GB
Logical volume LogVol00 successfully resized
Finally we need to resize the file system by typing resize2fs /dev/VolGroup00/LogVol00.
resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 20101888 (4k) blocks.
The filesystem on /dev/VolGroup00/LogVol00 is now 20101888 blocks long.
Type df -k to see if the new space is available to the logical volume.
df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
1713916000 82428400 88250280 48% /
/dev/sda1 101086 24538 71329 26% /boot
tmpfs 1037744 0 1037744 0% /dev/shm
The logical volume has now been resized and now has used space of just 48%!
At my job, after doing the same kind of procedure graph, we began to ask ourselves, why are using a LVM on a Linux VM guests?
ReplyDeleteSince we're no longer living in the physical OS world, we didn't need to use the OS hacks(LVM) to overcome physical disk limitations anymore.
We decided to Just let the hypervisor and virtual storage do that work for us.
For example, in our production setup (3 tier commerce with VMs for database , webserver, and appserver), we're see a great improvement in managability and performance (>10%) by just dropping LVM, and most partitions.
In your example, the resize process is 7 functional steps:
1. Increase size of VMDK
2. In VM OS, Create Partition (??)
3. REBOOT (!!)
4. PVCreate
5. VGExtend
6. LVExtend
7. Resize2fs
Going to a LVM/partition-less setup reduces expansion to 3 steps and we don't need to take the VM OS offline!
1. Increase size of VMDK
2- Inside the VM, OS, rescan the scsi drive with:'echo 1 >/sys/class/scsi_device//rescan; dmesg' (dmesg will check that you drive isize has grown)
3- Resize2fs.
Our current disk arrangement has 3 VM HD devices
0 - small device (100M) with a single BOOT partition
1 - entire device is /
2 - entire device is SWAP
Doing this has simplified resizing so much, I now let the junior admins and my manager expand drive space as needed.
It's also let's us really be spartan on space since expansion is so quick. Instead of increasing systems in 30-50GB chunks, we can do 10-15GB and let our rmonitoring system warn us when space gets tight.
Good comment, I agree them all. But this is just an article to let people have one option to expand disk size, and there are plenty ways to do same thing in both VM and Linux world though.
ReplyDeleteI used the method of Edward ( 7 steps) and I successfully expanded the disk of Linux EL5 guest from 150G to 350G for oracle appsR12.
ReplyDeleteI would like to try out with 3 step but I am truggling with resize2fs.
Could you please explain more about the use of resize2fs.
I have done:
$ echo "1" > /sys/class/scsi_device/0:0:0:0/device/rescan
$ dmesg
Extract from output of dmesg "SCSI device sda: 734003200 512-byte hdwr sectors (375810 MB)
"
$ resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.39 (29-May-2006)
The filesystem is already 38256640 blocks long. Nothing to do!
The volume is not bigger, check with :
$vgdisplay
Extract from output
" Alloc PE / Size 4796 / 149.88 GB
Free PE / Size 0 / 0
"
Thanks in advance
Please ignore my question, after reading carefully your post I have seen my mistake :
ReplyDeleteI am using LVM/partionned.
Dear Sir,
ReplyDeleteThanks very much for this very useful tutorial.
It has saved my bacon, in my Organisation.
In my VMware Course, I was told, that Thin provisioning would automatically increase the space allocated in and to the OS.
Imagine my surprise when my company database server crashed due to space limits :P
You are really a life saver.
Nice post, it was very usefull to me.
ReplyDeleteBut I found that instead of 3 steps, slightly more steps are involved (in a partition-less LVM setup):
1. Increase size of VMDK
2- Inside the VM, OS, rescan the scsi drive with:'echo 1 >/sys/class/scsi_device/<>/rescan; dmesg' (dmesg will check that you drive isize has grown)
3- resize PV, so that LVM recognizes the bigger disk:
pvresize /dev/sdb
4.Extend LV
5.resize FS
Cheers
On the slight chance that anonymous #1 reads this again, I would like to know what Linux distribution you are using? We have been testing the ability to grow a ext3 file system with no partitions and no LVM. We are trying to install RedHat 5 on /dev/sd# (with no partitions). It does not allow you to do this. We receive the error, "Device is a loopback and will need to be formatted before the install". To do some testing, I installed RedHat within partitions on my disks. Once the install is completed, I create a new file system on a standalone disk with no partitions (mkfs.ext3 /dev/sdd to be exact). I am able to then grow this partition exactly as you detailed above. However, I really want to know if there is a way to install RedHat on a partitionless system?
ReplyDeleteIf want to reply to me, email: jemurray@wustl.edu
Edward, thanks for posting this guide, it's helped a lot!
ReplyDeleteHello,
ReplyDeleteI think the post from Anonymous #1 mentions LVM/partition-less, which means (once again I think) that they're using the whole disk to run LVM on top of it. So instead of having /dev/sda1 as a Linux LVM partition and running 'pvcreate /dev/sda1', they probably didn't create a partition on sda, and used the whole disk for LVM ('pvcreate /dev/sda'), which then gives you the opportunity to increase the underlying VMDK, rescan the disk ('echo "1" > /sys/class/scsi_device/0:0:0:0/device/rescan' where 0:0:0:0 is the address of /dev/sda, as per 'ls -l /dev/disk/by-path'), and then increase the PV ('pvresize /dev/sda') after which you can increase the size of the LV you want to increase.
On the 'lvextend' front, if you want to use all the remaining space, instead of running 'lvextend -L +99GB', you can run 'lvextend -l +100%FREE' (and notice the different in case on the '-L' and '-l' options).
I hope this helps.
Although I have not tried this out ( but possibly will do that this week) the Comment #1 meant to create 3 virtual partitions from the vmware:
ReplyDelete0 - boot
1 - / and evrything beneeth
2 - SWAP
From the guest os perspsective this woul look like either /dev/sda, /dev/sdb, /dev/sdc or maybe /dev/sda1, /dev/sdb1, /dev/sdc1. Depending on that you create FS directly on to these "virtual" partitions without any LVM in-between !!!
When I test it, hopefully will rmember to post the results.
What do you mean by 3 partitions from the vmware? If you mean 3 VMDKs, and not using LVM, then I think you'll have to create a partition on each VMDK (which will appear each as a sd disk (/dev/sda, /dev/sdb and /dev/sdc), as I'm not sure you can use the whole disk to install linux directly without partition nor LVM.
ReplyDeleteSo, from a VMware point of view, the disks (VMDKs) are "virtual" disk, but as far as the guest OS is concerned, they are physical, SCSI disks. So you could use /dev/sda1 as /boot, /dev/sdb1 as the root filesystem, and /dev/sdc1 as the swap space. But I think you *will* need to create a Linux partition (type 83, from memory), for the /boot and / filesystems and Linux swap partition type (once again from memory, type 82).
Running CentOS under ESXi 4.1. This is the most understandable and straightforward procedure I found. Thanks very much!
ReplyDeleteOne reboot was required, after running fdisk, in order to update the partition table in the kernel.
This was fantastic. Thanks so much!
ReplyDeleteGood Post.
ReplyDeleteThe 3-step procedure described by anonymous one sounds good, but I think that's (at least with vmware workstation) is a one way road: One can increase the size of drives, but shrinking is no option (afaik). It would be of interest if this is also as easily as the described extension is possible
@ Paul Stone - you should not even need a reboot to update the partition table in the kernel, just re-read the partition table:
ReplyDeleteblockdev --rereadpt /dev/sdx
where x is the disk you've just changed.
@ Anonymous (September 7), with VMware, you can indeed grow disks, but not reduce them. The way around that, I found, is to create another (smaller) disk, add it to the VG you want to recude, and pvmove all the LVs to the new disk. At least that's how I did it.
* In VMware I add second disk to virtual machine
ReplyDelete* run pvscan or partprobe : see additional disk
* fdisk -l : list the disk
* fdisk /dev/sdb or sdb1 : to create new partition, take defaults (primary) and make 8e type (L)
* pvcreate /dev/sdb1 : create a new physical volume on the new partition
* vgdisplay to get LVM in this case was lvroot
* vgextend lvroot /dev/sdb1 : extend existing volume group (make sure vg free size is = new partition size using vgdisplay command)
* pvscan : to see verify new physical volume
* lvextend -L +4G /dev/mapper/lvroot-rootvg : extend existing LVM by adding new paritition or
* lvextend -l +100%FREE /dev/mapper/lvroot-rootvg : make the LV larger
* lvdisplay : make sure logical volume is increased
* resize2fs /dev/mapper/lvroot-rootvg : resize file system
@ Nick,
ReplyDeleteShould the "blockdev --rereadpt" command work even on /dev/sda (the boot device containing mounted filesystems)? The 'rescan' option for the '0:0:0:0' device worked so "fdisk -l" sees the larger disk size, but after I deleted and recreated the last partition on the disk (used for LVM PV), the blockdev command returns "BLKRRPART: Device or resource busy"
Thanks lol...
ReplyDeleteBelow mentioned link is very easy to understand,
http://www.redhatlinux.info/2010/11/lvm-logical-volume-manager.html
Thanks for this guide, I just extended one of our production servers.
ReplyDeleteThank you very much for this post!
ReplyDeleteI did have some trouble with extending the disk though. After some searching, I found out that I had to delete all my snapshots before I could extend my hdd.
Thanks again!
greatly helped, thanks a lot
ReplyDelete--Ldom
Great post! Thanks =)
ReplyDeletevmkfstools -X 200G "Your VMDK file here" working fine. I am able to see that space in fdisk -l /dev/sdX
ReplyDeleteBut when creating a partition I am not getting that additional space.
1) -X 110G makes the vmdk size to 110G . am i correct? Say I get 10G additional space by saying -X 110G on a 100G vmdk file. fdisk shows that its 110G , but when creating a partition I am not getting 10G???
2) From vsphere client I am not able to see the change in disk size.
Thanks for great HOWTO :)
ReplyDeleteThank you so much, it is very helpful.
ReplyDeleteAlright, since many are wondering about the first comment (I have, a lot) I found the way of making that setup work, it basicaly involves making a partition less install of your linux. Just tested it, resizing of / on /dev/sdc worked perfectly on production server.
ReplyDeleteComplete series of instructions found here: http://luke.bowbak.org/2013/01/24/partitions-thing-of-the-past-2.html , big fucking thanks to Luke Bakken.
Excellent steps ... very helpful! Thanks a million!
ReplyDeleteThanks for the process! Worked perfectly. And much easier to follow than reading dozens of pages of not very useful official Linux documentation.
ReplyDeleteGood day. I have VM of ubuntu 20.04. Today I extend /dev/mapper/xxx--lv-yyy--lg
ReplyDeleteIf I do: df -k I see: 153458192 42741768 103934212 30%
But vgdisplay shows the following: Total PE 38128
Alloc PE / Size 38128 / <148.94 GiB
Free PE / Size 0 / 0 !!! not found free space??
what's wrong?
Thank You and I have a neat give: House Renovation Canada house renovation montreal
ReplyDelete