Page 1 of 2

Database corruption [resolved]

Posted: Tue Nov 06, 2007 8:11 am
by Red Squirrel
Looks like the whole database corrupted itself, I have no clue how but I'm quite pissed off at the instability of that database system. Shows that converting to SQL is a much more important task and I should stop putting it off

I'm at work now so I can't do crap all. I'll see if I can figure it out on my lunch hour (which is more like 15 minutes by the time I get home and eat then go back to work) but doubt it, so it will have to wait till like 7:00pm tonight. :/

Sorry about this.

Archived topic from AOV, old topic ID:1610, old post ID:10156

Database corruption [resolved]

Posted: Tue Nov 06, 2007 8:37 am
by Red Squirrel
Ok I just did a blind database restore. I'm at work so I can't login to actually see how recent this backup is, someone let me know how recent it looks. If its super old or stuff is screwed up such as houses being gone, just let me know since I rather not run the shard if I'm only going to wipe everything after (to restore another backup, that is)

Archived topic from AOV, old topic ID:1610, old post ID:10159

Database corruption [resolved]

Posted: Tue Nov 06, 2007 8:59 am
by dprantl
Ok, so I just logged in and was 7/8th's of the way to killing dreadhorn. So I went ahead and killed him again and got out of there, then when the server saved it corrupted the DB AGAIN. And....

...

I got Crimson Cincture this time :evil: I can't believe that, I'm going to shoot myself :cry:

And I need to go to work :(

Archived topic from AOV, old topic ID:1610, old post ID:10161

Database corruption [resolved]

Posted: Tue Nov 06, 2007 9:06 am
by Red Squirrel
Wow is this ever fucked... I'll keep that backup knowing that its recent but definatly something is totally fucked, this will take a long time to fix since its impossible to pinpoint what is causing it.

If I can manage to reproduce this on TC1 it might be easier though. Was it the first world save after DreadHorn's death that did it?

Archived topic from AOV, old topic ID:1610, old post ID:10163

Database corruption [resolved]

Posted: Tue Nov 06, 2007 9:50 am
by dprantl
Red Squirrel wrote:Was it the first world save after DreadHorn's death that did it?
Yes.

Archived topic from AOV, old topic ID:1610, old post ID:10168

Database corruption [resolved]

Posted: Tue Nov 06, 2007 10:48 am
by Shadow
The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?

Archived topic from AOV, old topic ID:1610, old post ID:10169

Database corruption [resolved]

Posted: Tue Nov 06, 2007 11:42 am
by dprantl
There is no doubt that the death of dreadhorn is causing this. I bet if you restored the DB again and I just waited around with dreadhorn still alive for a world save, it would not corrupt. But I'm at work right now so I can't test this out.

Archived topic from AOV, old topic ID:1610, old post ID:10170

Database corruption [resolved]

Posted: Tue Nov 06, 2007 12:43 pm
by Red Squirrel
I can already confirm since there was a few saves before it crashed again.

Nice thing about help desk is if I have a RPD connection opened with a dos console, it looks like work. :twisted: :twisted:

I just tried something, we'll see if it works.

Everyone is free to go play again, but avoid taking out pets as if it crashes again you might loose them.


I have to go back to work now.

Archived topic from AOV, old topic ID:1610, old post ID:10171

Database corruption [resolved]

Posted: Tue Nov 06, 2007 1:14 pm
by dprantl
Although this will suck because by the time I get back from work, my dreadhorn will have timed out and I will be kicked out of there :evil: I will also not be able to test to see if killing him will reproduce the problem.

Archived topic from AOV, old topic ID:1610, old post ID:10172

Database corruption [resolved]

Posted: Tue Nov 06, 2007 1:50 pm
by Red Squirrel
I'll arrange for you to get another key given the circumstances. I just hope my 5 minute fix worked so I can actually concentrate on my work without this stress. :P

Archived topic from AOV, old topic ID:1610, old post ID:10173

Database corruption [resolved]

Posted: Tue Nov 06, 2007 2:22 pm
by Dumples
Red Squirrel wrote:I'll arrange for you to get another key given the circumstances. I just hope my 5 minute fix worked so I can actually concentrate on my work without this stress. :P
Don't stress. Your real life job is more important.

Archived topic from AOV, old topic ID:1610, old post ID:10174

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:30 pm
by Death
Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?

Archived topic from AOV, old topic ID:1610, old post ID:10175

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:47 pm
by dprantl
Seer Death wrote:
Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.

Archived topic from AOV, old topic ID:1610, old post ID:10178

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:51 pm
by dprantl
Actually Death, I can do it again right now if you give me the key (since I was kicked out while I was at work) with you watching if you want.

Archived topic from AOV, old topic ID:1610, old post ID:10179

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:52 pm
by Death
dprantl wrote:
Seer Death wrote:
Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.
I have a hunch.....did you teleport out of the room after the death or wait for the room to eject you? Also, did you disband your party before the room ejected you?

Archived topic from AOV, old topic ID:1610, old post ID:10180

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:53 pm
by dprantl
Seer Death wrote:
dprantl wrote:
Seer Death wrote:
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.
I have a hunch.....did you teleport out of the room after the death or wait for the room to eject you? Also, did you disband your party before the room ejected you?
I went over to the island with the healer and used the gate to go to gate library. I did not wait. There was no party.

Archived topic from AOV, old topic ID:1610, old post ID:10181

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:54 pm
by Death
hmmm okay. You were the only person in there then?

Archived topic from AOV, old topic ID:1610, old post ID:10182

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:54 pm
by dprantl
Seer Death wrote:hmmm okay. You were the only person in there then?
Yes. In fact, I was the only person online at the time, apart from Counselor Shadow.

Archived topic from AOV, old topic ID:1610, old post ID:10183

Database corruption [resolved]

Posted: Tue Nov 06, 2007 3:55 pm
by Death
dprantl wrote:
Seer Death wrote:hmmm okay. You were the only person in there then?
Yes. In fact, I was the only person online at the time, apart from Counselor Shadow.
Hmmm k that narrows things down a bit then.

Archived topic from AOV, old topic ID:1610, old post ID:10184

Database corruption [resolved]

Posted: Tue Nov 06, 2007 4:04 pm
by Red Squirrel
I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.

Archived topic from AOV, old topic ID:1610, old post ID:10185

Database corruption [resolved]

Posted: Tue Nov 06, 2007 4:14 pm
by dprantl
Well I can tell you both times it crashed I got recipes...

Archived topic from AOV, old topic ID:1610, old post ID:10187

Database corruption [resolved]

Posted: Tue Nov 06, 2007 4:17 pm
by Death
Red Squirrel wrote:I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.
Well seeing as we tested DH numerous times before and I tried to crash that part locally without success, I'm beginning to think that it is the new changes to the recipes as well. I'm trying to produce that locally right now.

Archived topic from AOV, old topic ID:1610, old post ID:10188

Database corruption [resolved]

Posted: Tue Nov 06, 2007 4:18 pm
by dprantl
Seer Death wrote:
Red Squirrel wrote:I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.
Well seeing as we tested DH numerous times before and I tried to crash that part locally without success, I'm beginning to think that it is the new changes to the recipes as well. I'm trying to produce that locally right now.
Me too, I just did dreadhorn on TC1 and have manually added a recipe into my pack. Let's see if that does it in during the save, unless you put the fix into TC1 Red?

Archived topic from AOV, old topic ID:1610, old post ID:10189

Database corruption [resolved]

Posted: Tue Nov 06, 2007 4:20 pm
by Red Squirrel
Yeah if fix is on live its also on TC1. I can't recall if I rebooted it though, think I did.

Archived topic from AOV, old topic ID:1610, old post ID:10190

Database corruption [resolved]

Posted: Tue Nov 06, 2007 6:13 pm
by Red Squirrel
I'm home now, so trying to reproduce it on TC1 to confirm that my fix is actually what fixed it.

I was SO not productive at work today because of this, just can't think straight when I know a critical server is down and I can't do anything about it. Then again its been so dead lately so I doubt it affected too many people, though it does make us look bad to have such long downtime. I kinda blew the 5 9's on this one. Then again, OSI has had lot of issues too so we're not alone.

Archived topic from AOV, old topic ID:1610, old post ID:10194