Skip to main content

Downtime and lost data

Posted by Kim on December 5, 2018, 9:55pm

Tonight was a hard night. The site was down for several hours for maintenance after our database went kerfluey. The issues we experienced were non-malicious (This was not an attack. No data was stolen.)

Most things came through the crisis just fine; so far, it looks like I was able to recover all groups, forum posts, private messages, kudos, accolades, custom templates, and purchase records.

Character and user pages DID take a hit, though. We have had to roll back character pages and users as much as ten days. If you created a character or account, or made changes in that time span, it's likely gone now. I can't tell you how upset I am that this has happened to our community, and how sorry I am.

I can say I'll move heaven and Earth to make sure this storm can NEVER happen again. I can also say I'll do everything I can to reunite people with lost data wherever it's possible - oddly, with your help, it may be possible re-link you to any RPs you were engaged in if you were doing them with an account or character fresher than the rollback. Read on for more details about that.

A LOT of data has been moved around to get us back up. There may be some unexpected results. PLEASE submit bug reports about anything that looks off to you to, no matter how small - you are never bothering me by bringing my attention to something that might be wrong, even if it turns out not to be a bug or problem! And you may well be helping me to help not just you, but many others, if you turn out to have spotted a clue to an issue. I really need your help to make sure everything is okay going forward!

Since user accounts were rolled back 10 days, I'm going through the purchase records and re-applying any Epicness or other perks you may have applied to your account in that time. This will take a few hours to sift through, and even so, you're still welcome to submit a bug report to me about it right away if you're worried!

If you lost a character or your user account in this awfulness, I could not be more sorry and upset on your behalf. The RPR is supposed to be here to support and celebrate your creations, not to hurt them. I want to do everything I can to at least re-unite you with the RPs you may have had.

If you have an email notification about a RP you were doing, whether in PMs or on the forums, please forward it to me at admin@rprepository.com with the name of your new account and and/or your re-created character and I will re-link that RP to your new account, and throw in some free stuff to try and make it up to you.


Don't have any emails about your RPs from us, but still know you had a RP that you want to be reunited with? Still email me with as many details as you can and I'll see what I can do. Include:
  • Your username
  • If applicable, your re-created character's page (link me!)
  • Any details you can remember about the RP you were doing. What forum was it on? What was the title of the PM conversation? What date did it start? That sort of thing!

I understand if you are feeling anger or sadness or even grief in this moment. I am feeling these things as well. It's an awful thing to happen and I am certain we all feel awful about it.

I want to thank everyone who sent me SO much support while I was working on this issue. You got me through it!

I will continue to update this space with any critical updates, as well as put out notices on our Facebook and Twitter feeds.

UPDATE 12:30am PST: I've restored hundreds of thousands of missing character pages. If you're still missing something more than 10 days old, definitely make sure you hit me up with those bug reports!

UPDATE 12:52am PST: If your account just adopted a character named "Zombie character," this is a placeholder for a lost profile. It is intended to give you back access into your private messages and groups you were participating in with that character. You may choose to delete the character, or to rename it and rebuild the profile on it.

UPDATE 1:30am PST: If you activated an item in the last ten days, the benefits should now be re-applied to your account. That is to say, your Epicness and character slots should be back.

Comments

Marie

December 5, 2018
10:48pm

Yeet! My characters came back after a short period of them being gone. Make sure you guys are filing the bug reports and giving them time to fix this. They've been working hard for hours. They haven't given up on fixing yet and they won't anytime soon ;)

MaighdeanDubh

December 5, 2018
10:47pm

and my one character's info is all gone...

AmongstTheStars

December 5, 2018
10:45pm

I'm happy you were able to get the site back! I'm missing one of my characters named "Jacob”. I was just going to inform you. Also I'm sorry all this is on your plate at the moment.
Best of wishes from all of us!

MaighdeanDubh

December 5, 2018
10:45pm

Okay, went through and I'm missing Artemisiah as well, but when I click on the 'view and sort characters' thing, she and my Aurora show up there.

Thank you for working so hard on resolving this <3

LadyOfGondor

December 5, 2018
10:44pm

Thank you so so much for all your hard work!! I would've given up and gone to bed LOL

Seriously though thank you so much!! Thank you for your hard work and for the updates!!

Queen

December 5, 2018
10:43pm

Thank you so much for all of the hard work that you did to save all that you could for us! I was so worried when I heard about what happened, but I'm so absolutely relieved that you saved as much as you did. Truly I'm sure that all of us are extremely grateful for all that you have done!

MercyInReach

December 5, 2018
10:38pm

Hey, if you lost any characters, I just figured out that if you have an editing link from that profile in your browsing history, click on it, it should take you to a page that says "rpr thinks this character isn't yours, send a bug report". Perhaps this will make it possible to link back to characters that got lost in oblivion?

-Danny-

December 5, 2018
10:38pm

I'm missing my character Austin Davis. If you can find him and revive him would be appreciated. If not I can remake him later. Thank you for doing what you can to help us Kim. <3

Degu

December 5, 2018
10:35pm

Anyone who lost anything, double check because a lot of folks are saying their pages all just came back. <3

KansasVenomoth

December 5, 2018
10:34pm

Oddly enough, I didn't lose any characters, but rather I gained some back... ah, well, all's well that ends well.

Thanks for the hard work in getting this back online, Kim. You're a lifesaver.

MercyInReach

December 5, 2018
10:31pm

Thank you for working so hard to get this figured out and salvage as much as possible.
I lost a massive character slot that I had some serious info dumped into for my book, but it's my fault for not backing up to google docs before I went to bed...

As for my other 21 characters, I'm missing all of their widgets and info - but I already put in a bug report. I hope it can be salvaged, as I've been adding and adding to their profiles for almost three years and it would take months to rewrite all of them with all of the same or close to the same information.

Regardless, you are a serious trooper and while this is all very disheartening and I'm trying not to have a panic attack at 1am, I can only imagine how hard this has been for you.

Degu

December 5, 2018
10:30pm

I don't know what you're doing Kim but it's all coming back!! i thought i'd lost so much

MaighdeanDubh

December 5, 2018
10:30pm

So far it looks like one of my characters was lost, but in doing a search on her name, Aurora Metaxes, she shows up in the search, but she's not listed in my dashboard.

yamashta

December 5, 2018
10:26pm

I believe all of my characters were made more than ten days ago and all are entirely blank or missing most of their pages' content. I have sent in a bug report.

Good luck restoring what can be restored- for everyone. This is a very disheartening event for many of us.

Dylan

December 5, 2018
10:26pm

A massive thanks to all the hard work you put in, especially with the continuous updates as to the status around the downtime. Thank you for that and putting such a tremendous amount of work into it.

This is also a super good lesson for folks to always have multiple backups and not rely on one source.

ToaNuva

December 5, 2018
10:25pm

About two of my characters were lost and another two had their info wiped clean in this mess. But I hope everything will be good in a few days so that we all can enjoy this site equally.

LakotaSiouxWarrior

December 5, 2018
10:25pm

80 % of my characters blanked out and I lost 3.

xKeatonx

December 5, 2018
10:24pm

Well I had 5 characters blanked out, which will takes hours of rewriting. However, Sin's was saved, meaning I have the format to rewrite them all. I will bring up that 4 of the characters wiped haven't been touched in weeks, so that might be something we can fix. But PLEASE get some sleep or something Kim. Thanks for working so hard for us ^~^

UsagiSmith

December 5, 2018
10:21pm

Oh man! I got really scared when this happen, but I am glad that everything is fine now. :)

LakotaSiouxWarrior

December 5, 2018
10:19pm

I lost 3 characters I'll make new ones. Thanks for the hard work you did to get the site going again. Go get some rest. RPR is the best roleplaying website ever.