Forums

Hello,

I'm trying to back up a large file (sqlite db file) - its 4.5GB. I have obtained a shared lock on the db and attempted to copy the file from another shell ... I get the following errors ...

cp: error writing ‘/home/danhartropp/test/backend/data/data.db’: Input/output error

cp: failed to extend ‘/home/danhartropp/test/backend/data/data.db’: Input/output error

Is this likely to be due to the size of the file, or is there something else going on?

Thanks

Dan

Hi Dan, it is likely that cp tries to put the whole db into RAM, which is causing your process to look like it is going crazy and so is getting killed. Perhaps you could try rsyncing?

Basically just rsync source-file target-file instead of cp source-file target-file

See here for more info.

We did have a couple of file server glitches around that time today, so it may be worth just retrying?

Hi guys,

Tried cp again this morning - same error as before. rsync also failed ... error message is below. Immediately after I tried rsync, the whole backend (dashboard, forums etc) dropped out for a couple of minutes, with a 502-backend error.

The rysnc error message was

rsync: write failed on "/home/danhartropp/test/backend/data/data.db": Input/output error (5) rsync error: error in file IO (code 11) at receiver.c(389) [receiver=3.1.0]

Please can someone have a look at this as a matter of urgency ... I'm getting nervous that I am unable to back up my data.

Dan

Hi Dan, that does sound like there may be an issue. We'll take a look.

In the meantime, we might be able to do the backup for you manually. Do you want us to make a copy of that file and put it somewhere?

In the medium term, you might consider a migration to a different database as well? I think that, at a db size of several gigs, sqlite may not be the right answer... Especially when it's stored on an NFS filesystem, as it will be on PA...

Thanks - if you could move it manually, that would be great. I'm trying to copy ~/sixandtwothrees/backend/data/data.db to ~/test/backend/data/data.db

Moving it all into mysql is on the to-do list in the next couple of months, although performance has generally been good - its being used for read only queries on a low traffic site.

Dan

Hi danhartropp. I have rsynced the data.db across. You should probably double check that it's not corrupted or anything (eg: say there was user session data being written to your db during the rsync)

Thanks - I can see the file, but the sqlite shell can't open it ... could it be the file permissions?

Dan

Yes- stupid of me. Just changed it. Check now?

ps: did you have a sinking feeling when you couldn't open it initially? :p hope you feel better now!

Looks good to me. I've run a few test queries from the shell and they seem to be working.

To be honest, the sinking feeling has been with me since I deleted a random selection of records in the test db yesterday afternoon (which is the reason I then tried to copy the live db in the first place).

Any idea what caused the problem in the first place?

Dan

hmm- we will be investigating it...