Database corruption [resolved]

Stay up to date with shard happenings
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

Looks like the whole database corrupted itself, I have no clue how but I'm quite pissed off at the instability of that database system. Shows that converting to SQL is a much more important task and I should stop putting it off

I'm at work now so I can't do crap all. I'll see if I can figure it out on my lunch hour (which is more like 15 minutes by the time I get home and eat then go back to work) but doubt it, so it will have to wait till like 7:00pm tonight. :/

Sorry about this.

Archived topic from AOV, old topic ID:1610, old post ID:10156
Honk if you love Jesus, text if you want to meet Him!
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

Ok I just did a blind database restore. I'm at work so I can't login to actually see how recent this backup is, someone let me know how recent it looks. If its super old or stuff is screwed up such as houses being gone, just let me know since I rather not run the shard if I'm only going to wipe everything after (to restore another backup, that is)

Archived topic from AOV, old topic ID:1610, old post ID:10159
Honk if you love Jesus, text if you want to meet Him!
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Ok, so I just logged in and was 7/8th's of the way to killing dreadhorn. So I went ahead and killed him again and got out of there, then when the server saved it corrupted the DB AGAIN. And....

...

I got Crimson Cincture this time :evil: I can't believe that, I'm going to shoot myself :cry:

And I need to go to work :(

Archived topic from AOV, old topic ID:1610, old post ID:10161
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

Wow is this ever fucked... I'll keep that backup knowing that its recent but definatly something is totally fucked, this will take a long time to fix since its impossible to pinpoint what is causing it.

If I can manage to reproduce this on TC1 it might be easier though. Was it the first world save after DreadHorn's death that did it?

Archived topic from AOV, old topic ID:1610, old post ID:10163
Honk if you love Jesus, text if you want to meet Him!
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Red Squirrel wrote:Was it the first world save after DreadHorn's death that did it?
Yes.

Archived topic from AOV, old topic ID:1610, old post ID:10168
Shadow
Posts: 205
Joined: Sat Oct 27, 2007 1:33 am

Database corruption [resolved]

Post by Shadow »

The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?

Archived topic from AOV, old topic ID:1610, old post ID:10169
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

There is no doubt that the death of dreadhorn is causing this. I bet if you restored the DB again and I just waited around with dreadhorn still alive for a world save, it would not corrupt. But I'm at work right now so I can't test this out.

Archived topic from AOV, old topic ID:1610, old post ID:10170
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

I can already confirm since there was a few saves before it crashed again.

Nice thing about help desk is if I have a RPD connection opened with a dos console, it looks like work. :twisted: :twisted:

I just tried something, we'll see if it works.

Everyone is free to go play again, but avoid taking out pets as if it crashes again you might loose them.


I have to go back to work now.

Archived topic from AOV, old topic ID:1610, old post ID:10171
Honk if you love Jesus, text if you want to meet Him!
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Although this will suck because by the time I get back from work, my dreadhorn will have timed out and I will be kicked out of there :evil: I will also not be able to test to see if killing him will reproduce the problem.

Archived topic from AOV, old topic ID:1610, old post ID:10172
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

I'll arrange for you to get another key given the circumstances. I just hope my 5 minute fix worked so I can actually concentrate on my work without this stress. :P

Archived topic from AOV, old topic ID:1610, old post ID:10173
Honk if you love Jesus, text if you want to meet Him!
User avatar
Dumples
Posts: 484
Joined: Fri Jan 05, 2007 4:57 pm

Database corruption [resolved]

Post by Dumples »

Red Squirrel wrote:I'll arrange for you to get another key given the circumstances. I just hope my 5 minute fix worked so I can actually concentrate on my work without this stress. :P
Don't stress. Your real life job is more important.

Archived topic from AOV, old topic ID:1610, old post ID:10174
Image
User avatar
Death
Posts: 7919
Joined: Thu Sep 30, 2004 10:12 pm

Database corruption [resolved]

Post by Death »

Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?

Archived topic from AOV, old topic ID:1610, old post ID:10175
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Seer Death wrote:
Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.

Archived topic from AOV, old topic ID:1610, old post ID:10178
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Actually Death, I can do it again right now if you give me the key (since I was kicked out while I was at work) with you watching if you want.

Archived topic from AOV, old topic ID:1610, old post ID:10179
User avatar
Death
Posts: 7919
Joined: Thu Sep 30, 2004 10:12 pm

Database corruption [resolved]

Post by Death »

dprantl wrote:
Seer Death wrote:
Shadow wrote:The world save was approximately, idk, 5 mins after the Dreadhorn drop that Altar did. Maybe the Dreadhorn spawn overloaded the server somehow? How often is Dreadhorn killed?
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.
I have a hunch.....did you teleport out of the room after the death or wait for the room to eject you? Also, did you disband your party before the room ejected you?

Archived topic from AOV, old topic ID:1610, old post ID:10180
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Seer Death wrote:
dprantl wrote:
Seer Death wrote:
Dreadhorn has been tested at least 7 times without this problem occuring. Of course, there were a few recent script modifications but it's just a cliloc which wouldn't do much. Would somebody care to explain who they were using to kill dreadhorn, and what exactly happened?
I was killing dreadhorn. I used the same character as when we did it together. I did not see anything out of the ordinary until the save right after dreadhorn's death. I was already out of there and in my house. Then Red reverted to a backup in which I was almost done killing dreadhorn, so I killed him again and left again, and as soon as a save happened, the corruption happened again.
I have a hunch.....did you teleport out of the room after the death or wait for the room to eject you? Also, did you disband your party before the room ejected you?
I went over to the island with the healer and used the gate to go to gate library. I did not wait. There was no party.

Archived topic from AOV, old topic ID:1610, old post ID:10181
User avatar
Death
Posts: 7919
Joined: Thu Sep 30, 2004 10:12 pm

Database corruption [resolved]

Post by Death »

hmmm okay. You were the only person in there then?

Archived topic from AOV, old topic ID:1610, old post ID:10182
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Seer Death wrote:hmmm okay. You were the only person in there then?
Yes. In fact, I was the only person online at the time, apart from Counselor Shadow.

Archived topic from AOV, old topic ID:1610, old post ID:10183
User avatar
Death
Posts: 7919
Joined: Thu Sep 30, 2004 10:12 pm

Database corruption [resolved]

Post by Death »

dprantl wrote:
Seer Death wrote:hmmm okay. You were the only person in there then?
Yes. In fact, I was the only person online at the time, apart from Counselor Shadow.
Hmmm k that narrows things down a bit then.

Archived topic from AOV, old topic ID:1610, old post ID:10184
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.

Archived topic from AOV, old topic ID:1610, old post ID:10185
Honk if you love Jesus, text if you want to meet Him!
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Well I can tell you both times it crashed I got recipes...

Archived topic from AOV, old topic ID:1610, old post ID:10187
User avatar
Death
Posts: 7919
Joined: Thu Sep 30, 2004 10:12 pm

Database corruption [resolved]

Post by Death »

Red Squirrel wrote:I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.
Well seeing as we tested DH numerous times before and I tried to crash that part locally without success, I'm beginning to think that it is the new changes to the recipes as well. I'm trying to produce that locally right now.

Archived topic from AOV, old topic ID:1610, old post ID:10188
dprantl
Posts: 1048
Joined: Wed Feb 07, 2007 11:41 pm

Database corruption [resolved]

Post by dprantl »

Seer Death wrote:
Red Squirrel wrote:I think the cause was recipes, it generates an instance of the artifact then deletes it on the fly in order to get the hue, itemid and properties for display purposes. I had it set to delete the item on save, which turned out to corrupt the DB.

At least t hats my theory, I'm not 100% sure it was that.
Well seeing as we tested DH numerous times before and I tried to crash that part locally without success, I'm beginning to think that it is the new changes to the recipes as well. I'm trying to produce that locally right now.
Me too, I just did dreadhorn on TC1 and have manually added a recipe into my pack. Let's see if that does it in during the save, unless you put the fix into TC1 Red?

Archived topic from AOV, old topic ID:1610, old post ID:10189
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

Yeah if fix is on live its also on TC1. I can't recall if I rebooted it though, think I did.

Archived topic from AOV, old topic ID:1610, old post ID:10190
Honk if you love Jesus, text if you want to meet Him!
User avatar
Red Squirrel
Posts: 29209
Joined: Wed Dec 18, 2002 12:14 am
Location: Northern Ontario
Contact:

Database corruption [resolved]

Post by Red Squirrel »

I'm home now, so trying to reproduce it on TC1 to confirm that my fix is actually what fixed it.

I was SO not productive at work today because of this, just can't think straight when I know a critical server is down and I can't do anything about it. Then again its been so dead lately so I doubt it affected too many people, though it does make us look bad to have such long downtime. I kinda blew the 5 9's on this one. Then again, OSI has had lot of issues too so we're not alone.

Archived topic from AOV, old topic ID:1610, old post ID:10194
Honk if you love Jesus, text if you want to meet Him!
Locked