Copying a disk containing a btrfs filesystem
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one. I can use cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.
If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/
Michael Schuerig posted on Thu, 10 Apr 2014 15:21:01 +0200 as excerpted:
Quoted 13 lines
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one. I can use cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.
Quoted 3 lines
If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
It's not a btrfs tool, but several of the tools you mentioned aren't, and you didn't mention dd (or ddrescue, if your source device starts giving you issues while you're cloning). Using it you'd clone the entire raw device, including any not yet allocated areas, in a straight-across bit- perfect copy. Of course you'd need a target of either the same size or larger in ordered to do so...
That should give you a bit-perfect copy including the filesystem UUIDs, etc, which will confuse btrfs if you try to mount anything btrfs with both devices attached, so don't. Do your clone, then umount and disconnect the old device before trying to mount the new one.
In fact, there are entire purpose-built special live-image distributions such as Clonezilla for managing storage devices like this.
Or use a more general purpose "rescue" live-image distro such as SysRescue:
Both of these tools support MS Windows systems as well. Image-cloning is of course OS-agnostic -- what's on the cloned image doesn't matter, just whether the source device is readable and the destination writable. And sysrescue is often used in the MS world to reset lost passwords and etc. I used to use it for that back when I still worked on friends' MS systems for "gratis" (well, traded for a good dinner or whatever), before I decided that wasn't a worthwhile use of my time, tho I'd be happy to teach them about Linux if they wanted, or might still do it if they paid me enough, but then there's other computer repair services that will do that if they're willing to pay.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 1 line · On Thursday 10 April 2014 13:58:45 Duncan wrote:
Michael Schuerig posted on Thu, 10 Apr 2014 15:21:01 +0200 as
excerpted:
Quoted 24 lines
Quoted 13 lines
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one. I can use cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.
Quoted 3 lines
If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
It's not a btrfs tool, but several of the tools you mentioned aren't, and you didn't mention dd (or ddrescue, if your source device starts giving you issues while you're cloning). Using it you'd clone the entire raw device, including any not yet allocated areas, in a straight-across bit- perfect copy. Of course you'd need a target of either the same size or larger in ordered to do so...
Quoted 5 lines
That should give you a bit-perfect copy including the filesystem UUIDs, etc, which will confuse btrfs if you try to mount anything btrfs with both devices attached, so don't. Do your clone, then umount and disconnect the old device before trying to mount the new one.
Thanks for pointing out dd and especially ddrescue, which I didn't know. I regularly use dd to copy images onto USB thumb drives, but I'm a bit wary regarding its use on hard disks. I've always been concerned that if I get another disk, not necessarily of the same make, that has nominally the same capacity, it still might be a few blocks smaller. Well, I just found one that has the right size, apparently.
Thanks again,
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/
I would see one (dangerous? risky? needing more options perhaps?) solution:
dd if=/dev/sda of=/dev/sdb
/dev/sda: old disk/dev/sdb: new disk
Maybe there is another much more complicated solution:
Plug the old disk in a USB dock/case, do the same for the new disk in another dock/case, plug both docks/cases in another linux system running recent kernel and btrfs-progs and convert to a "-dconvert=raid1 -mconvert=raid1" profile with a balance. Then degrade it by removing the old disk and rebalance-convert back to single or DUP profile (i am not quite sure this is even possible).
Just an idea. I wouldn't trust me.
Quoted 28 lines · Michael Schuerig
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one. I can use cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.
If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
Michael
--
Michael Schuerig
mailto:[email protected]
http://www.schuerig.de/michael/
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
George Eleftheriou posted on Thu, 10 Apr 2014 16:00:06 +0200 as excerpted:
Quoted 10 lines
Maybe there is another much more complicated solution:
Plug the old disk in a USB dock/case, do the same for the new disk in another dock/case, plug both docks/cases in another linux system running recent kernel and btrfs-progs and convert to a "-dconvert=raid1 -mconvert=raid1" profile with a balance. Then degrade it by removing the old disk and rebalance-convert back to single or DUP profile (i am not quite sure this is even possible).
Just an idea. I wouldn't trust me.
Creative idea, actually, and one I've seen come up here before tho if you read my earlier response I didn't mention it myself, as I failed to recall it.
But yes, it is somewhat more complicated, and sort of an abuse of existing features for a purpose they weren't intended for. It should still work and such unintended purpose uses are a good thing, but because it is unintended usage, it won't have been tested much if at all, and given that btrfs isn't itself fully stable yet, there's a fair likelihood that some unanticipated complexities would arise while doing it, as well.
Meanwhile, by the very fact that btrfs is /not/ yet entirely stable, there's even more reason than normal to have good, tested backups. Given that, there should be no harm in trying this, since even if it somehow kills the working copy, one can still restore from those already tested backups. And someone's gotta be the first to test this method and report on how it went, right?
Meanwhile (2), given the existence of those tested backups, there's yet another way to accomplish things. Simply restore from the backups the same way you would if the working copy went down and you had to restore it, only restore to the new device instead of the old one. =:^)
(Actually, given my usual insistence on this key point, that btrfs isn't yet fully stable and that as a result by definition, you either have tested backups or you're demonstrating by practice that you simply don't care about losing the data anyway whatever your words might otherwise claim, I really should have thought of and mentioned the restoring from backups method first, instead of having it occur to me as an after- thought. I must be getting sloppy and careless in my thinking! =:^( )
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 5 lines · On Thursday 10 April 2014 15:15:02 Duncan wrote:
Meanwhile (2), given the existence of those tested backups, there's yet another way to accomplish things. Simply restore from the backups the same way you would if the working copy went down and you had to restore it, only restore to the new device instead of the old one. =:^)
As the OP, let me insist that I have multiple backups. However and unfortunately, those backups do not contain the snapshots that I'd like to preserve when exchanging the disk. What makes the case complicate is not the question how to preserve and copy the current data; it's how to retain the historic data embodied in snapshots.
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/
Quoted 4 lines
What makes the case complicate is
not the question how to preserve and copy the current data; it's how to retain the historic data embodied in snapshots.
You can always rsync (incrementally with --link-dest) to "another place" the sequence of snapshots, provided of course there is enough space in this "other place". However I understand this may not be the coolest thing to do especially taking into account the SMART warnings for imminent deterioration.
Quoted 3 lines · Duncan
claim, I really should have thought of and mentioned the restoring from backups method first, instead of having it occur to me as an after- thought. I must be getting sloppy and careless in my thinking! =:^( )
Maybe this is related to the fact that btrfs is maturing, gradually causing you a sense of complete safety ;-)
Quoted 9 lines · On Thursday 10 April 2014 18:01:31 George Eleftheriou wrote:
Quoted 3 lines
What makes the case complicate is
not the question how to preserve and copy the current data; it's how to retain the historic data embodied in snapshots.You can always rsync (incrementally with --link-dest) to "another place" the sequence of snapshots, provided of course there is enough space in this "other place". However I understand this may not be the coolest thing to do especially taking into account the SMART warnings for imminent deterioration.
In the case at hand, I'll go with ddrescue. As far as backups are concerned, rsync --link-dest may be an option, however even at best it would only give file-granularity COW as opposed to block-granularity. I'm not sure if this would work for me in terms of disk space consumption.
I *think* send/receive with clone sources might be the key to a solution. I'm still hoping that someone with a far better understanding of btrfs than me gives it a try first...
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/
Michael Schuerig posted on Thu, 10 Apr 2014 18:24:00 +0200 as excerpted:
Quoted 3 lines
I *think* send/receive with clone sources might be the key to a solution. I'm still hoping that someone with a far better understanding of btrfs than me gives it a try first...
Well, there's both that (which I don't know enough about to discuss in any intelligent way at all), and the fact that at least until very (very) recently (like recent enough I'm not sure it'll all be in 2.15 yet), btrfs send/receive still had some serious corner-case bugs, such that to my knowledge if it worked without error it was reliable, there were numbers of cases (like where it was originally subdir /a/b/, and then switched to /b/a, reversing the nesting) where it would spit out errors, leaving the receive in a half-received state.
So it worked for some people but not others, and for some it would work for awhile, then would start erroring out, whenever they hit whatever corner-case.
Regulars on this list will have seen all the recent work going into send/ receive bug tracing and fixing, however, which will of course ultimately make it far more reliable than it had been. Surely it's already far more reliable, as a fair number of those corner cases have now been found and dealt with.
But what I do NOT know yet, is where in the process we are, in terms of /reliable/ send/receive. Relatively, we're certainly closer than we were, but did that move use from say 80% to 90% reliable, or from 80% to 97 or 98% reliable (which is about where I'd put general btrfs at this point), or from 90% to 98-ish% reliable, or... ?
Without that knowledge, even if you did know the specific CL options you needed, I'd still consider it very much a "great if it works, but don't count on it until you actually see it complete without errors, and even then, be aware that the next time you try it could error out, because it's still very much in bug-fixing-phase" solution.
Meanwhile, the ddrescue solution is mature technology, proven to work with many different types of filesystems over many years, now, so that's the basket I'd be putting my resource eggs into at this point. =:^)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Am Donnerstag, 10. April 2014, 17:51:26 schrieb Michael Schuerig:
Quoted 7 lines · On Thursday 10 April 2014 15:15:02 Duncan wrote:
Quoted 5 lines
Meanwhile (2), given the existence of those tested backups, there's yet another way to accomplish things. Simply restore from the backups the same way you would if the working copy went down and you had to restore it, only restore to the new device instead of the old one. =:^)
Quoted 5 lines
As the OP, let me insist that I have multiple backups. However and unfortunately, those backups do not contain the snapshots that I'd like to preserve when exchanging the disk. What makes the case complicate is not the question how to preserve and copy the current data; it's how to retain the historic data embodied in snapshots.
Well I think this is scriptable – although this would be some work.
Like this – pseudo shell code without verifying exact argument order:
- for each subvolume create according subvolume in destination
- for each snapshot – from oldest to newest:
- rsync / btrfs send snapshot
- snapshot with same name
- rsync next newer snapshot
I have snapshots on backup drive… so I´d probably ditch snapshots on production side… but if you want them… thats another idea.
Although I like the balance idea as well. I used it to RAID-1 my /home to dual SSD setup. But I didn´t to the degrading step.
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Dne Čt 10. dubna 2014 15:21:01, Michael Schuerig napsal(a):
Quoted 8 lines
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one.
Yes it does
You can make the old disk a seeding device and use it to seed the new one like this:
btrfstune -S 1 <old_device>
mount <old_device> /mnt
# this will be mounted read-only
btrfs dev add <new_device> /mnt
mount -o remount,rw /mnt
btrfs dev delete <old_device>
According to my experiments, the filesystem on the new device will have different UUID, so if you are mounting using UUIDs, you must change it everywhere, but other than that the new filesystem should have the same content as the old one (including subvolumes).
Quoted 11 lines
I can use
cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
Michael
Quoted 7 lines · On Thursday 10 April 2014 19:17:13 Jan Kouba wrote:
Dne Čt 10. dubna 2014 15:21:01, Michael Schuerig napsal(a):
Quoted 6 lines
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
Quoted 6 lines
Quoted 2 lines
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one.
Yes it does
You can make the old disk a seeding device and use it to seed the new one like this:
Intriguing. I didn't find much on the topic, but these pages seem pertinent
http://en.wikipedia.org/wiki/Btrfs#Seed_devices
https://btrfs.wiki.kernel.org/index.php/Seed-device
http://unix.stackexchange.com/a/97819/64935
Quoted 1 line
btrfstune -S 1 <
old_device>
Does the "1" mean "true" or is this indeed an integer-value arg? I've only seen the "1" in examples, are there other possible values?
Quoted 8 lines
mount <
old_device> /mnt# this will be mounted read-onlybtrfs dev add <
new_device> /mntmount -o remount,rw /mnt
btrfs dev delete <
old_device>
Is the last one the step where data is copied from the seed device to the second device? No balance required before deleting the device?
Thanks!
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/
Dne Čt 10. dubna 2014 20:36:32, Michael Schuerig napsal(a):
Quoted 19 lines · On Thursday 10 April 2014 19:17:13 Jan Kouba wrote:
Quoted 7 lines
Dne Čt 10. dubna 2014 15:21:01, Michael Schuerig napsal(a):
Quoted 6 lines
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
Quoted 8 lines
Quoted 2 lines
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one.
Yes it does
You can make the old disk a seeding device and use it to seed the new
one like this:
Intriguing. I didn't find much on the topic, but these pages seem pertinent
Quoted 8 lines
http://en.wikipedia.org/wiki/Btrfs#Seed_devices
https://btrfs.wiki.kernel.org/index.php/Seed-device
http://unix.stackexchange.com/a/97819/64935Quoted 1 line
btrfstune -S 1 <
old_device>Does the "1" mean "true" or is this indeed an integer-value arg? I've only seen the "1" in examples, are there other possible values?
Yes, "1" means "true", "0" false. If you run it twice with the same argument, it says something like "Seeding flag already set/unset.".
Quoted 4 lines
Quoted 4 lines
mount <
old_device> /mnt# this will be mounted read-only
Quoted 8 lines
Quoted 5 lines
btrfs dev add <
new_device> /mntmount -o remount,rw /mnt
btrfs dev delete <
old_device>Is the last one the step where data is copied from the seed device to the second device? No balance required before deleting the device?
Yes, everything is copied, no balance needed.
Well actually if you don't need to preserve the filesystem on the old device, you can just add the new device into the filesystem and remove the old one like this:
mount <old_device> /mnt
btrfs dev add <new_device> /mnt
btrfs dev delete <old_device> /mnt
The last command will copy everything from <old_device> to the <new_device> and somehow wipe the <old_device> so it can't be mounted anymore.
This method will also preserve the UUID of the filesystem so there is no need to change any config files.
Jan Kouba
Hi, Michael
Btrfs send/receive can transfer incremental snapshots as well - you're looking for the "-p" or "parent" parameter. On the other hand, it might not be the right tool for the job.
If you're 100% happy with your old disk's *content*/layout/etc (just not happy with the disk's reliability), try an overnight/over-weekend ddrescue instead:
http://www.forensicswiki.org/wiki/Ddrescue
What I've done in the past is scripted ddrescue to recover as much data as possible. Its like using dd between two disks except that it can keep a log of bad sectors that can be retried later. The log also helps by ensuring that if you cancel the operation, you can start it again and it will continue where it left off.
Additionally, you can have it skip big sections of the disk when it comes across bad sectors - and it can "trim" these sections on subsequent runs.
Btrfs send/receive has the advantage that you can run it while your system is still active. DDrescue has the advantage that it is very good at recovering 99% of your data where a disk has lots of bad sectors.
For btrfs, send/receive the main subvolumes then, afterward, send/receive the snapshots using the "parent" parameter, "-p". There *is* the possibility that this needs to be reversed - as in, the backup should be treated as the parent instead of the other way around:
btrfs send /home | btrfs receive /mnt/new-disk/home btrfs send -p /home /backups/home-2014-04-08 | btrfs receive/mnt/new-disk/backups/.
____
Below is from the last scriptlet I made when I last did the ddrescue route (in that case I was recovering a failing NTFS drive). It was particularly bad and took a whole weekend to recover. The new disk worked flawlessly however. :)
How I've used ddrescue in the past is to connect the failing and new disk to a server. Alternatively, using USB, you could boot from a rescue CD/flash drive and do the rescue there.
a) Identify the disks in /dev/disk/by-<whatever-you-choose> and put those values into the bash script below. ensure that it refers to the disk as a whole (not a partition for example). This ensures that re-ordering of the drives after a reboot won't affect the process.
b) Set up a log file location on a separate filesystem - a flash drive is ideal unless you've gone the "server" route, where I normally just put the log into a path on the the server as so: /root/brendan-laptop.recovery.log
#!/bin/bash
src_disk=/dev/disk/by-id/ata-ST3250410AS_6RYF5NP7
dst_disk=/dev/disk/by-id/ata-ST3500418AS_9VM2ZZQS
#log=/path/to/log
log=~/brendan-laptop.recovery.log
#Sector size (default is 512 - newer disks should probably be 4096) sector_size=4096
#Force writing to a block device - disable if you're backing up to an image file #force=""
force="-f"
# We want to skip bigger chunks to get as much data as possible before the source disk really dies # For the same reason, we also want to start with (attempting) to maintain a high read rate #Minimum read rate in MB/s before skipping
min_readrate=10485760
#Default skip size is 64k.
for skip_size in 65536 16384; do
#Going forward
ddrescue -r 1 -a $min_readrate -b $sector_size -d $force -K $skip_size
$src_disk $dst_disk $log
#Going in reverse
ddrescue -r 1 -a $min_readrate -b $sector_size -d $force -K $skip_size
-R $src_disk $dst_disk $log
done
#Default skip size is 64k.
#This time re-trying all failed/skipped sections
for skip_size in 4096; do
#Going forward
ddrescue -r 1 -a $min_readrate -b $sector_size -d $force -K $skip_size
-A $src_disk $dst_disk $log
#Going in reverse
ddrescue -r 1 -a $min_readrate -b $sector_size -d $force -K $skip_size
-R -A $src_disk $dst_disk $log
done
#Default skip size is 64k.
for skip_size in 1024 256 64 ; do
#Going forward
ddrescue -r 1 -b $sector_size -d $force -K $skip_size $src_disk
$dst_disk $log
#Going in reverse
ddrescue -r 1 -b $sector_size -d $force -K $skip_size -R $src_disk
$dst_disk $log
done
echo "Done. Run an chkdsk/fsck/whatever-might-be appropriate for the new disk's filesystem(s)"
Quoted 19 lines · Michael Schuerig
SMART indicates that my notebook disk may soon be failing (an unreadable/uncorrectable sector), therefore I intend to exchange it. The disk contains a single btrfs filesystem with several nested(!) subvolumes, each with several read-only snapshots in a .snapshots subdirectory.
As far as I can tell, btrfs currently does not offer a sensible way to duplicate the entire contents of the old disk onto a new one. I can use cp, rsync, or send/receive to copy the "main" subvolumes. But unless I'm missing something obvious, the snapshots are effectively lost. btrfs send optionally takes multiple clone sources, but I've never seen an example of its usage.
If that's what "experimental" means, I'm willing to accept it. However, I'd like to emphasize that there's still something missing. Of course, most of all I'd like to be proved wrong.
Michael
Signature · 4 lines
__________ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97
Quoted 3 lines · On Friday 11 April 2014 12:39:31 Brendan Hide wrote:
If you're 100% happy with your old disk's *content*/layout/etc (just not happy with the disk's reliability), try an overnight/over-weekend ddrescue instead
Thanks again to everyone who replied and especially for suggesting ddrescue. In the meantime, I've got a suitable new disk and copied the data to it using ddrescue (it took about 12 hours for 1TB). The disk had one fault extending over a few successive sectors.
For some reason, I had to repair the first partition, which I use for swap. The large second with btrfs on it seems fine. I ran btrfs scrub on it and it found one error:
Apr 16 09:48:27 fuchsia kernel: [ 6792.829186] btrfs: checksum error at
logical 74443923456 on dev /dev/sdb2, sector 145398288, root 1228, inode 59102093, offset 929792, length 4096, links 1 (path:usr/lib/debug/.build-id/8f/b82df57b7b6fff7033f6abd7de914b82f98160.debug)
That file is part of a "historical" snapshot which I have deleted by now and thus presumably dealt with the problem.
Michael
Signature · 3 lines
Michael Schuerig mailto:[email protected] http://www.schuerig.de/michael/