Skip to main content

Downtime and lost data

Posted by Kim on December 5, 2018, 9:55pm

Tonight was a hard night. The site was down for several hours for maintenance after our database went kerfluey. The issues we experienced were non-malicious (This was not an attack. No data was stolen.)

Most things came through the crisis just fine; so far, it looks like I was able to recover all groups, forum posts, private messages, kudos, accolades, custom templates, and purchase records.

Character and user pages DID take a hit, though. We have had to roll back character pages and users as much as ten days. If you created a character or account, or made changes in that time span, it's likely gone now. I can't tell you how upset I am that this has happened to our community, and how sorry I am.

I can say I'll move heaven and Earth to make sure this storm can NEVER happen again. I can also say I'll do everything I can to reunite people with lost data wherever it's possible - oddly, with your help, it may be possible re-link you to any RPs you were engaged in if you were doing them with an account or character fresher than the rollback. Read on for more details about that.

A LOT of data has been moved around to get us back up. There may be some unexpected results. PLEASE submit bug reports about anything that looks off to you to, no matter how small - you are never bothering me by bringing my attention to something that might be wrong, even if it turns out not to be a bug or problem! And you may well be helping me to help not just you, but many others, if you turn out to have spotted a clue to an issue. I really need your help to make sure everything is okay going forward!

Since user accounts were rolled back 10 days, I'm going through the purchase records and re-applying any Epicness or other perks you may have applied to your account in that time. This will take a few hours to sift through, and even so, you're still welcome to submit a bug report to me about it right away if you're worried!

If you lost a character or your user account in this awfulness, I could not be more sorry and upset on your behalf. The RPR is supposed to be here to support and celebrate your creations, not to hurt them. I want to do everything I can to at least re-unite you with the RPs you may have had.

If you have an email notification about a RP you were doing, whether in PMs or on the forums, please forward it to me at admin@rprepository.com with the name of your new account and and/or your re-created character and I will re-link that RP to your new account, and throw in some free stuff to try and make it up to you.


Don't have any emails about your RPs from us, but still know you had a RP that you want to be reunited with? Still email me with as many details as you can and I'll see what I can do. Include:
  • Your username
  • If applicable, your re-created character's page (link me!)
  • Any details you can remember about the RP you were doing. What forum was it on? What was the title of the PM conversation? What date did it start? That sort of thing!

I understand if you are feeling anger or sadness or even grief in this moment. I am feeling these things as well. It's an awful thing to happen and I am certain we all feel awful about it.

I want to thank everyone who sent me SO much support while I was working on this issue. You got me through it!

I will continue to update this space with any critical updates, as well as put out notices on our Facebook and Twitter feeds.

UPDATE 12:30am PST: I've restored hundreds of thousands of missing character pages. If you're still missing something more than 10 days old, definitely make sure you hit me up with those bug reports!

UPDATE 12:52am PST: If your account just adopted a character named "Zombie character," this is a placeholder for a lost profile. It is intended to give you back access into your private messages and groups you were participating in with that character. You may choose to delete the character, or to rename it and rebuild the profile on it.

UPDATE 1:30am PST: If you activated an item in the last ten days, the benefits should now be re-applied to your account. That is to say, your Epicness and character slots should be back.

Comments

Mina

December 5, 2018
10:18pm

These things happen! Thanks so much for keeping us up to date on twitter.

Legion

December 5, 2018
10:18pm

You did an amazing job saving everything you could!

natjust

December 5, 2018
10:16pm

two* not weight

natjust

December 5, 2018
10:16pm

I lost weight characters as well, no idea if I can get them back. The odd thing is one of them was my first character and much older then 10 days. The other was newer, and I don't think they were less then 10 days created, all I've maybe done in that timeframe is add a new image for him. Any possible way of finding out what happened to them?

UrbanLady

December 5, 2018
10:11pm

Two of my characters were deleted, but thank you for your efforts. <3

Xpoint_MetaPulse

December 5, 2018
10:11pm

...Scratch that. Asami's thumbnail is broken and all my profiles are blanked.
I don't mind, as...well, I may be reinventing a few of my units for Season 3 of my RPs. I'll just roll with it.

Zelphyr

December 5, 2018
10:09pm

Thank you for your hard work, Kim! I'll try to keep alert, and hopefully I don't miss something over not remembering it right.

Skipbab

December 5, 2018
10:07pm

It erased three of mine, then wiped the last four completely clean, Except only a single page on one character about his built-in antagonist.

Postey

December 5, 2018
10:07pm

Kim you are a hero <3 Thank you for all your hard work!

Mytho

December 5, 2018
10:07pm

Sadly only one of my characters was spared, however thank you none the less for the hard work to get everything else back in order. Time to get to working hard on retyping all my babies profiles!

Hades_

December 5, 2018
10:05pm

Way to go kim! Thank you for all your hard work and dedication to this site and the community. Please get some rest. You seriously need it after all of this mess.

Kim

December 5, 2018
10:04pm

Thanks everyone!

And another reminder: If you're missing characters that you made more than 10 days ago, please send me an immediate bug report.

If you see a friend complaining that they are missing older characters, get them to submit a bug report with details of who/what they're missing

Cinnawolf

December 5, 2018
10:03pm

I seem to have lost a lot more than 10 days back. I am missing over half, to 80% of my character pages that i know are far older than 10 days...

Grotty

December 5, 2018
10:02pm

It weirdly picked and chose which of my characters it would erase. Unfortunately the ones with the most work. ;-;

CrimsonKnight

December 5, 2018
10:01pm

Thank you for your hard work, Kim! You're a superstar, and no one has any right to be angry for something that was out of your control. Thank you for everything you do for us!

Derglet

December 5, 2018
10:01pm

This was a crazy little thing to sit through haha! Thanks for being so on top of this and providing updates as quickly as you could, Kim! Your devotion is appreciated.

Copper_Dragon

December 5, 2018
10:00pm

I know I've watched the frantic work you were doing both through social media and private channels, so congratulations are absolutely in order for getting the site back up so swiftly again, Kim!

And a huge kudos to the RPR community for hanging in there and being so understanding and supportive during this rocky evening! <3 You guys are all fantastic.

Xpoint_MetaPulse

December 5, 2018
10:00pm

...10 Unread.
Looks like my data loaded fine. I'll keep ya posted.