On the Diamant/Incan Gold Merge

In March 2017, I sent out a note to many BGG users indicating that I was going to merge the game pages for Diamant and Incan Gold. Anyone who had rated either game or who had either game listed as being in their collection should have received this note.

As I understand it, when Incan Gold was first added to the database in 2006, it was on the same game page as Diamant, then after the Eagle-Gryphon version of Incan Gold was released in 2009 the game was split into its own page due to it being "different enough" from Diamant thanks to five new artifact cards. IELLO released a new version of the game in 2016 that included the artifacts but reverted to the original Diamant name. Yay for the game being available again in a nice version, but boo for the confusion this new version caused, with English, French, German, and other versions of IELLO's edition being added to both the Diamant and Incan Gold pages.

Given that the two games were 95% the same, I decided to merge them together again, rewriting the game description on the Diamant page to cover all versions of the game. I sent a note explaining this decision to users, then a second note when folks asked about the status of their collection listings and logged plays:

Quote:

My apologies, but I wanted to send a follow-up to yesterday's note about the pending merge of Incan Gold into Diamant. Ideally this will be my last note to all who have this game in their collection in some manner!

Someone sent me the following note: "Eric, I've played and rated both games. What will happen to my ratings (2 of them, rating comments, and plays?) I assume one of each will go away; which?"

Thanks for asking something along these lines! Comments and ratings carry over and duplicate in the combined listing. After all, right now you can add multiple copies of a game to your collection and rate/comment on each one separately. That feature already exists on a single listing, and the merge preserves those distinctions.

To double-check this, I created three game listings from scratch, rated each differently, and put a different comment on each one. I linked only one of those ratings/comments (the third one) to a particular version of the game; the other two I just placed on the game itself. (Did you know that you link a rating/comment to a particular version of a game? This site has lots of hidden features like that.)

I then merged all of these game listings and initially I was worried as a click on the ratings page showed me this:

You can see my separate 9, 7 and 5 ratings, but the comment is from the game I rated a 5, while the rating was from the game I gave a 7. Not sure why the data is presented that way, but that's something I need to ask the tech guys about.

Seeing only one of my ratings/comments worried me initially — but then I noticed the "reset filters" link under the "Ratings & Comments" header. Clicking that link showed me this:

Apparently the system automatically hides multiple ratings/comments from the same user under the default setting. This keeps you from being overwhelmed with duplicates. (I looked at Fluxx Promo Cards, for example, and after resetting filters, I see that someone has rated this item a 10 more than thirty times. If you go from that page to another part of the site, then come back to the Fluxx Promo Card page, the filters are automatically applied again, hiding all but one of this person's 10 ratings.)

I created a fourth game listing to test play counts since I forgot that the first time, and play counts get merged together with no distinction between one version of the game and another. Play count data is preserved, so if you took notes on the location, other players, etc, then that data carries over into the combined listings, but you can't specify which version of a game you play other than to add a comment along those lines. (This isn't a failure of the merge as much as a lack of a feature in the "play count recording" set-up; you can add details to play counts if you want to preserve a distinction as to which version of a game you played.)

Hope this answers any lingering questions...

I know some folks care a lot about these types of details, so I wanted to make sure that everything would work right, and that indeed seemed to be the case.

Well...

After I sent my note, I did nothing related to this merger for months. I headed to one con after another, with preparations for each taking time and BGG News posts taking time and this merger not being at the top of my to-do list. I have lots of projects like this — things that would be great to have done, but not things that are urgent to do. Perhaps you have lots of projects like this as well. In any case, someone reminded me of the merge-that-wasn't in November 2017 ahead of BGG.CON, so I finally did it.

Now, something you might not know about merges is that in years past when you merged something, you clicked a button, then after a short bit received a message on the page that said "Merge complete". At some point, I stopped receiving that message and just saw a 504 message instead. I might have reported this error message to our programmers Scott and Dan, but knowing me, I probably didn't; after all, the things had indeed been merged, so everything must be working okay, right? This must be just a nonsense error message.

So I clicked merge on the Incan Gold page, with Diamant being the target. The description of the target game is what remains after a merge; that game's title remains the representative title, the cover image the representative image. All of the version information of the title being merged gets added to the versions that already exist. The images get added to the gallery of existing images.

After I received the 504 message, I went back to the Diamant page and discovered that it still had only twelve versions instead of the twenty or so I had been expecting. (I needed to merge versions as well following the game merge. Again, doing so would preserve user collection info.) Perhaps I was mistaken and I hadn't clicked "merge". Perhaps the page timed out. I waited a minute, reloaded, and still saw only twelve versions, so I merged Incan Gold again.

That was a mistake.

Most of the things that I merge are new or obscure. Admins approve two submissions for a new game, for example, not realizing what the other person is doing, so I merge the listings. A user realizes that this little-known German game from the 1980s is actually the same as this little-known French game from the 1980s, so I merge them.

Diamant and Incan Gold fall into the category of neither new nor obscure. Thousands of users had rated each game and had them listed in their collections. I thought the merge had failed, but it was still going on in the background despite me having received the now-customary 504 message. When I clicked merge again, we entered new ground in terms of what the software was trying to do, with simultaneous merges happening at the same time. This wasn't supposed to be possible, but hey, there we were, merging things simultaneously.

What seems to have happened is that while the first merge was in the process of happening, the second merge looked at the user data for Incan Gold and saw nothing (because it was being rewritten as Diamant data or whatever it is that happens inside those computing boxes), so it decided it was done with collection info, and it raced through the version data and *boom* the merge was finished — not complete, mind you, but finished.

Eventually I saw twenty-something versions on the Diamant page, so I merged all the identical ones, and things seemed to be okay. A couple of days later, I saw a post asking about a user's logged plays for Incan Gold since they were missing. Collection info was missing as well. Scott and Dan were already at BGG.CON, so I resolved to talk with them about the problem later. Now it's later, and after learning what I did they're (1) ensuring that it's impossible in the future to have two merges running simultaneously, (2) researching exactly what I broke, (3) figuring out how to recover user data on Incan Gold, although it seems like they'll have to reach back to May 2017 in order to do so (due to how back-ups are handled), and (4) placing me in an oubliette to prevent further database disruptions.

To those affected by this issue, I apologize for the trouble. Ideally Scott and Dan will be able to undo what needs undoing. As for me, I'll refrain from merging older, established items under threat of losing my hands. I've already seen one disaster, so I don't want to encounter a second one...