ROM Hack playhistory.dat timestamp

[holr]

Hello, I am trying to decode the playhistory.dat (00010022) file, after reviewing information such as PTM_Services on 3dbrew, and editing-activity-log-on-emunand.388212 here on gbatemp (I cannot post URLs at the present time), I am at a loss how the 4 byte value actually equates to a timestamp. Tried seconds/milliseconds since epoch (linux, 1970/1/1, 3ds 2001/1/1) but the values are erroneous. Has anyone ever been able to decode the timestamp values in playhistory.dat successfully?

 

[smudgethefirst]

Has anyone ever been able to decode the timestamp values in playhistory.dat successfully?

So, I think I figured it out, or at least I'm getting very close. But it's bizarre, and I'm hoping someone can help me make sense of it.

Basically, here are the steps for interpeting PlayHistory.dat's timestamp values:

1. Read the 4 byte value as an Integer¹
2. Multiply it by 3.75
3. Add it to 946684800 (Unix timestamp for the year 2000 epoch)
4. Treat that as your Unix timestamp (seconds since 1970) and feed it into your standard date/time library

...I only figured this out because:

• I've had a 3DS since 2012.
• I started parsing Pedometer.dat and could tell which years and months I was active.
• I accidentally set my system datetime to 2042 and then set it back a little while later, so I had entries in the future, in addition to entries for much of 2012-2023.
• I cross-correlated those Pedometer.dat entries with PlayHistory.dat's values (based on times where I started and stopped playing), and the ratio between PlayHistory's value and the correct date's "seconds since year 2000" was always roughly 3.75, which makes no sense to me, but there you go.

What I'd like to know is whether it is actually 3.75, or something very close to 3.75. I can't quite tell, but the above algorithm gets me down to what I think is the exact day for most of these entries, but I don't know if it's the exact hour/minute/second.

———
¹I think unsigned 32-bit is fine, but none of the dates I've encountered would have a "1" in the signed bit.

 

[TheOkster]

So, I think I figured it out, or at least I'm getting very close. But it's bizarre, and I'm hoping someone can help me make sense of it.

Basically, here are the steps for interpeting PlayHistory.dat's timestamp values:

1. Read the 4 byte value as an Integer¹
2. Multiply it by 3.75
3. Add it to 946684800 (Unix timestamp for the year 2000 epoch)
4. Treat that as your Unix timestamp (seconds since 1970) and feed it into your standard date/time library

...I only figured this out because:

• I've had a 3DS since 2012.
• I started parsing Pedometer.dat and could tell which years and months I was active.
• I accidentally set my system datetime to 2042 and then set it back a little while later, so I had entries in the future, in addition to entries for much of 2012-2023.
• I cross-correlated those Pedometer.dat entries with PlayHistory.dat's values (based on times where I started and stopped playing), and the ratio between PlayHistory's value and the correct date's "seconds since year 2000" was always roughly 3.75, which makes no sense to me, but there you go.

What I'd like to know is whether it is actually 3.75, or something very close to 3.75. I can't quite tell, but the above algorithm gets me down to what I think is the exact day for most of these entries, but I don't know if it's the exact hour/minute/second.

———
¹I think unsigned 32-bit is fine, but none of the dates I've encountered would have a "1" in the signed bit.

How were you able to get PlayHistory.dat out of the 00000000 container in /data/<16 Digit ID>/sysdata/00010022?

Post automatically merged: Feb 26, 2023

Never mind, I was able to get data simply using pld.dat in the Activity Log save file. However, I didn't have to multiply by 3.75 to find the Unix Timestamp.
Just to show:
My very first Devil Survivor: Overclocked (0x00880300) US entry was on May 10, 2021 and my first 0x00880300 entry in the dat was:
00 88 03 00 00 00 04 00 40 4D 2C 28 3C 00 00 00

40 4D 2C 28 converts to 673992000 when interpreted as Int32 with the little endian byte order. When added to the unix timestamp for Jan 1 2000, I got 1620676800 which is on May 10, 2021.

Post automatically merged: Feb 26, 2023

Also, it seems like the four bytes after the timestamp is the number of seconds played in that entry.
The value provided is always a multiple of 60 so I wonder why they decided to store it in seconds rather than minutes.

 

[smudgethefirst]

How were you able to get PlayHistory.dat out of the 00000000 container in /data/<16 Digit ID>/sysdata/00010022?

I used 3ds-save-tool (disa-extract.py) to extract the files out of the container file.

I also threw together the attached gm9 script to make it easier to repeatedly get the file off of the device.

Edit: Yeah, it's true that pld.dat also has the data, and the timestamp values in there are easier to read -- but my understanding is that PlayHistory.dat is the system's source of truth for that data, and pld.dat only syncs when you open the Activity Log app. (Activity Log's pld.dat may also have a smaller limit on the amount of data it holds, but I haven't actually confirmed whether that is the case yet.)

 

[TollyH]

I've gone through my PlayHistory.dat file and I think I've mostly figured out the format.
First off, the timestamp value isn't measuring seconds since 2000-01-01, it's measuring minutes. The reason you seemingly needed to multiply the timestamp by 3.75 is because the lower 4 bits of the timestamp aren't part of the timestamp at all - they're 4 flags that tell you what the event is representing. Right shifting those 4 bits off gets you a division by 16, and 60 / 16 = 3.75. This also explains why the logged events were seemingly out of order sometimes.

As for what the bits mean normally (least significant bit to highest):

• Bit 1 = Whether the event represents an opening of something or the closing. 0 = open, 1 = close
• Bit 2 = Whether the app in question is a HOME menu applet (e.g. the menu itself, game notes, etc.) or a full application (e.g. any game, system settings, etc). 0 = full app, 1 = applet
• Bit 3 = Whether the event corresponds to a resume/suspend or a launch/quit (suspend here means going to the HOME menu without quitting an app). 0 = launch/quit, 1 = resume/suspend
• Bit 4 = Only ever set when TitleID is equal to FFFFFFFFFFFFFFFF. Part of a set of alternate meanings with that TitleID.

Entries with TitleID FFFFFFFFFFFFFFFF seem to be some sort of special logging system, and change the meaning of the 4 bits. They should be interpreted as one number here, not as flags. Here's all of the combinations of 4 bits with the special logs that I can find in my file:

• 0000 = DS Mode start
• 0001 = DS Mode end
• 0111 = unknown, occurs just before "SAFE_MODE System Updater" launches it seems, only occurs once in my whole file
• 1000 = sleep mode start
• 1001 = sleep mode end
• 1010 = 3DS services stopped (happens at shutdown and also when launching DS Mode apps)
• 1011 = start of system clock change (stores original time)
• 1100 = end of system clock change (stores new time)

All of this should be enough information to extract all the information that the Activity Log does. I've written up my findings on 3dbrew at 3dbrew.org/wiki/PTM_Savegame with extra information for those interested.

 

[DaniElectra]

I've gone through my PlayHistory.dat file and I think I've mostly figured out the format.
First off, the timestamp value isn't measuring seconds since 2000-01-01, it's measuring minutes. The reason you seemingly needed to multiply the timestamp by 3.75 is because the lower 4 bits of the timestamp aren't part of the timestamp at all - they're 4 flags that tell you what the event is representing. Right shifting those 4 bits off gets you a division by 16, and 60 / 16 = 3.75. This also explains why the logged events were seemingly out of order sometimes.

As for what the bits mean normally (least significant bit to highest):

• Bit 1 = Whether the event represents an opening of something or the closing. 0 = open, 1 = close
• Bit 2 = Whether the app in question is a HOME menu applet (e.g. the menu itself, game notes, etc.) or a full application (e.g. any game, system settings, etc). 0 = full app, 1 = applet
• Bit 3 = Whether the event corresponds to a resume/suspend or a launch/quit (suspend here means going to the HOME menu without quitting an app). 0 = launch/quit, 1 = resume/suspend
• Bit 4 = Only ever set when TitleID is equal to FFFFFFFFFFFFFFFF. Part of a set of alternate meanings with that TitleID.

Entries with TitleID FFFFFFFFFFFFFFFF seem to be some sort of special logging system, and change the meaning of the 4 bits. They should be interpreted as one number here, not as flags. Here's all of the combinations of 4 bits with the special logs that I can find in my file:

• 0000 = DS Mode start
• 0001 = DS Mode end
• 0111 = unknown, occurs just before "SAFE_MODE System Updater" launches it seems, only occurs once in my whole file
• 1000 = sleep mode start
• 1001 = sleep mode end
• 1010 = 3DS services stopped (happens at shutdown and also when launching DS Mode apps)
• 1011 = start of system clock change (stores original time)
• 1100 = end of system clock change (stores new time)

All of this should be enough information to extract all the information that the Activity Log does.

That is some great documentation. Awesome!