“No space left on device” – kdump generation issue (CentOS/RHEL)
Unable to generate kdump and fails with the following error:
Checking for memory holes : [100.0 %] | copy_bitmap_file: Can't write the bitmap(/tmp/kdump_bitmapQijuck). No space left on device" kdump: dump target is /dev/[ 14.793505] EXT4-fs (dm-0): re-mounted. Opts: (null) mapper/VGExaDb-kdump_vol kdump: saving to /kdumproot/kdump/u01/crashfiles/127.0.0.1-2020-04-29-09:46:31/ kdump: saving vmcore-dmesg.txt kdump: saving vmcore-dmesg.txt complete kdump: saving vmcore The kernel version is not supported. The makedumpfile operation may be incomplete. Checking for memory holes : [100.0 %] | copy_bitmap_file: Can't write the bitmap(/tmp/kdump_bitmapQijuck). No space left on device create_2nd_bitmap: Can't copy 1st-bitmap to 2nd-bitmap. makedumpfile Failed. kdump: saving vmcore failed touch: cannot touch '/kdumproot/u01/crashfiles/kdump_post.log': No such file or directory tee: /kdumproot/u01/crashfiles/kdump_post.log: No such file or directory 2020-04-29 09:46:38 +0200 [INFO] Started kdump post tee: /kdumproot/u01/crashfiles/kdump_post.log: No such file or directory
Kdump initramfs unable to fetch real disk in order to dump the core.
Due to huge memory system need some place for makedumpfile to work and mark included/excluded memory pages. Add –work-dir to the makedumpfile command which will use actual disk rather than kdump ramdisk.
1. Edit the /etc/kdump.conf file as below and add –work-dir=/kdumproot/kdump/u01/crashfiles entry.
# vi /etc/kdump.conf core_collector makedumpfile -p -d 31 --work-dir=/kdumproot/kdump/u01/crashfiles
2. Restart kdump service:
# systemctl restart kdump
Note that path found for –work-dir came from below message:
kdump: saving to /kdumproot/kdump/u01/crashfiles/127.0.0.1-2020-04-29-09:46:31/