Dedoose’s Black Eye, Crash, and Recovery Efforts

As many of you are aware, Dedoose services came crashing down Tuesday, May 6th at around 4:30 pm PST.

First, in short, after we complete the final stage of data restoration, the vast majority of Dedoose projects will not have been affected.  Our deepest regret is for those projects which will have been directly impacted and we are prepared to work with any of you on a personal case-by-case basis to remedy and assist in any way we can.

This devastating system ‘collision’ of Tuesday night resulted from an series of events leading to the failure of Dedoose services running on the Microsoft Azure platform.  To be clear, Dedoose services failed, not Azure.  In short, work done on one aspect of Dedoose led to the failure of another, cascading to pull down all of Dedoose.  The timing was particularly bad because it occurred in the midst of a full database encryption and backup.  This backup process, in turn, corrupted our entire storage system. Our immediate work with Microsoft support did not result in any substantial recovery.  Here’s where we are and where we’re going:

  1. The first restoration of Dedoose services took place Wednesday morning with Dedoose services restored, and the database up, accurate, and current as of the first week in March
  2. Initially we hesitated to bring the system back online at the March state but didn’t want to hang up any users whose data were not affected
  3. Any data, new codes and coding, etc. that any users add to your projects  from this point forward will not be disrupted as we bring past back-ups on-board, so all are free to continue working in the system
  4. A second data recovery and restoration is taking place as I write.  We have high confidence that data added to Dedoose up to mid-April will be recovered and restored.  Stay tuned; we will broadcast on this shortly as progress is made on our blog and Facebook
  5. Again, most devastating, and despite our work with Microsoft, is that we are not optimistic we will be able to recover data added to the system for roughly the 2 – 3 week window preceding  the failure on Tuesday May 6 at approximately 4:30 pm PST
  6. Once the second restoration is completed, we will all be better able to assess the state of each project.  Where needed, and if at all possible, we are prepared to dig in and offer our assistance to help bring things fully up-to-date where at all possible
  7. We will post widely when the second restoration is completed and we’ll be ready to work from there to address any remaining issues
  8. We’ve already implemented certain protections on the Azure deployment to prevent such an event re-occurring

And, for the more technically curious, we’re going tour de force on protection by:

  1. Deploying a database mirror/slave in Azure
  2. Deploying a database mirror/slave into Amazon S3
  3. Keeping a mirror copy of the entire blob storage including all file data, backups, video data synchronized nightly to our private server in an encrypted volume
  4. Storing nightly database backups on the VHD, Azure Blob Storage, and Amazon S3 Storage
  5. Mirroring all Azure file data into an Amazon S3 bucket
  6. Carrying out a weekly restore exercise for the database backups to ensure integrity
  7. Carrying out monthly bare bones from nothing restores of the entire Dedoose Platform to ensure integrity

This is an unprecedented and completely unanticipated event in the history of Dedoose services which went on line in 2006.  Regrettably, and as high as our technical guard was, we nonetheless did not recognize the danger and, as such, SCRC is fully responsible to every user, regardless of any issues with our vendors.   As you can imagine, this is a painful and challenging moment for us and we are doing all we can to help restore and re-build user data and, certainly, confidence.  We are here if you have any further questions and will be posting widely as we make progress.  As you might imagine, we are overwhelmed at the moment with user inquiries and working as fast as we can to respond.

We thank you for your patience as we work to return Dedoose to an even stronger platform, and truly hope you’ll stay around so we can continue to serve your research needs and regain your full confidence.


Eli Lieber

President | CEO, SocioCultural Research Consultants, LLC (

13 thoughts on “Dedoose’s Black Eye, Crash, and Recovery Efforts

  1. Arnold Toutant

    I am still unable to get into my account to see if anything is there on my project. It tells me that my password or user name is wrong. Then, when I try to change the Password, I get the error message again. I keep being put into a runaround.

    1. DedooseSCRC Post author

      If you created your account between March 3rd and April 11th, then please wait until the second restore is live. We will email you when that restore is up so you can go ahead and check.

      1. Erica

        Any idea of a ballpark when the second restore will go live? Tomorrow? Next week? Next month?? Any information would be helpful at this point. You need to post more often than once every couple of days due to the seriousness of what happened.

    2. Lynne Huffman

      I continue to have the same problems with being unable to login. Any possible solutions yet?

  2. Rebecca Pietrzykowski

    I understand how hard you are working, but my dissertation advisors do not. I am defending my research next weekend! The entire dissertation is due May 30th. After that, I would need to pay $3000.00 for an extension. Please, if you can, let me know if this is going to be possible. A lot of what I did was during the third and fourth weeks of April. Please, please let me know my status. I need to know if I should be recoding at this time.

  3. Adam Kruse

    You will never regain my full confidence. What I would like to regain is my full payment.

  4. Katherine Gold

    You seriously have LOST almost 4 weeks of our user work and data? How in the *&&^%%$# world did you not plan for daily backups of your system? We never ever ever would have paid to use your system in the first place had we known that we could actually LOSE our work and it could not be restored (post April 11 work). This is truly unacceptable.
    –primary investigator

  5. Deborah Yeoh-Wang

    The vast majority of all the files I coded for my thesis have been deleted. I had just under 40 files coded with almost 2000 tags. Most of those files were created prior to mid April. I’m down to 9 files with a total 219 tags. My thesis is due in two weeks, and I don’t have time to recode everything. When will the second data recovery be completed?

  6. kelley jones

    Help please – I too trusted Dedoose and switched from Atlas ti – I have strict dissertation deadline – graduation depends upon this. all my data is gone after February! this is insane. I can’t redo this and I worked daily between my full time job in Dedoose. I can’t locate anything even from march. what can I expect? I am distraught and completely panicked.

  7. Amanda

    I created a free trial account on March 30 (Sydney, Australia time) but paid up for a one year subscription on May 6. How will I be affected? Will I be able to see my work up until April 11? Will the system recognise that I’ve paid for an account now?

    I can’t log on and the system doesn’t recognise my user name or email. When will we get more information about the second restore? I’m very time limited and trying to decide to start all over again with another analysis tool or to wait it out for Dedoose.

  8. John Horton

    We lost all but one data entry during the crash. When will you be getting us some more information about the recovery of the data through April? This has caused us great distress. We have lost a tremendous amount of work effort and are about to miss a grant deadline due to lack of data to include in our application. We are reaching desperation to get some news.


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>