View Issue Details

IDProjectCategoryView StatusLast Update
0002454The Dark ModGUIpublic26.07.2021 04:00
Reportertels Assigned Toduzenko  
PrioritynormalSeverityfeatureReproducibilityN/A
Status assignedResolutionreopened 
Product VersionTDM 1.06 
Target VersionTDM 2.10Fixed in VersionTDM 2.10 
Summary0002454: Implement CC (closed captions, subtitles)
Descriptionndk already created a proof-of-concept, we just need to integrate it in our code.
TagsNo tags attached.

Relationships

related to 0002779 confirmed Translate the menu and HUD into different languages 
has duplicate 0001858 closedtels ConversationSystem needs support for subtitles 

Activities

nbohr1more

nbohr1more

06.08.2018 14:24

developer   ~0010739

http://forums.thedarkmod.com/topic/11752-subtitles/
duzenko

duzenko

03.07.2021 18:34

developer   ~0014146

Last edited: 03.07.2021 18:40

View 2 revisions

At revision: 9434 (code)
At revision: 16317 (assets)
Pending the review only the briefing video is affected
Known issues: .srt file must be outside of .pk4, line breaks probably broken on Linux
And I'm not sure if I broke the CMake build too?
stgatilov

stgatilov

14.07.2021 11:26

administrator   ~0014153

Discussion on dev forums:
  https://forums.thedarkmod.com/index.php?/topic/20992-subtitles-support/
Basically, more work is needed =)
stgatilov

stgatilov

25.07.2021 13:56

administrator   ~0014181

Picked up the torch =)

Here are some generic changes, which I used in subsequent subtitles-related code:
  r9501. idStr: Added Split with list of string delimiters, and SplitLines like in Python.
  r9502. idBinSearch: support custom comparator, support heterogeneous lookup.
  r9503. idCinematic can now return filename/path of the video.

The library for parsing SRT was pretty awful, like that guy tried to add as much files/classes as he could, even though they were empty =)
So I reimplemented SRT parsing (it is a trivial format):
  r9504. Replaced SubtitleParser library with hand-written function, moved subtitles from idSoundChannel to idSoundSample.

The major chunk of architectural changes comes next:
  r9505. Expanded support of subtitles: in-game sounds, several subtitles at once, fromVideo sounds.
  r16327. Fixed subtitles in briefing videos, added subtitles overlay GUI for in-game.
At this point, subtitles should work both in-game and in briefing video.

Then it turned out that we'd better require mentioning subtitles in sound shader, so here is one more architectural change:
  r9506. Now sound shader must explicitly mention subtitles in order to load them.

Finally, some minor things:
  r9507. Sync subtitles to video if sound is played from it.
  r16328. Added subtitles setting into main menu GUI.
stgatilov

stgatilov

25.07.2021 14:16

administrator   ~0014182

In order to enable subtitles, one has to:
  1) Write "subtitle story" or "subtitle speech" or "subtitle effect" in sound shader.
  2) For every x/y/z.ogg sound file mentioned in the shader, add x/y/z.srt file with subtitles (same for ,wav).
If sound is loaded from video x/y/z.mp4 ("fromVideo" keyword), then put subtitles into file x/y/z.srt.
Note that "subtitle" keyword must be BEFORE referencing sound files or video.

The second word in "subtitle" line means verbosity level.
  "story" is for FM-specific story-related text, which player should be able to understand.
  "speech" is for regular speech, e.g. guards saying something generic on patrol, or crying words in combat.
  "effect" is for non-speech sounds, I'm not even sure yet that we need this level...
Depending on cvar tdm_subtitles, non-important subtitles can be filtered out.
For instance, by default only "story" subtitles are displayed, but player can raise it to "speech" level in the menu.
stgatilov

stgatilov

25.07.2021 14:25

administrator   ~0014183

Last edited: 25.07.2021 14:26

View 2 revisions

The general architecture is like this.

When sound shader is parsed, we look for "subtitle" keyword.
If it is present, then we try to load .srt file for every sound sample loaded (idSoundSample::LoadSubtitles).
Note that the situation when .srt is missing is silently ignored.
The subtitles data is stored in idSoundSample.

When sound thread mixes sounds, it calls idSoundWorldLocal::AddChannelContribution for every active idSoundChannel.
In this method we fetch active subtitles with idSoundChannel::GatherSubtitles, and store them into active idSoundWorld object.
The sound world has two sets of subtitles data: one currently being filled from sound thread, another one returned to everyone who wants to know the currently active subtitles (switching between sets is guarded by mutex).

The GUI object has method idUserInterfaceLocal::UpdateSubtitles, which fetches active subtitles from the sound world and updates gui::subtitle0, gui::subtitle1, etc. variables.
We call it for:
  1) main menu when it is active
  2) subtitles overlay during game
As the result, these GUIs can easily display subtitles.
Note that we have several subtitle slots in GUI (for showing several parallel conversations at once), and UpdateSubtitles method distributes active subtitles into slots.
duzenko

duzenko

25.07.2021 20:05

developer   ~0014184

Wow, guys, it sure feels good to see an 11-year old ticket closed just like this
Thank you @stgatilov, @nbohr1more
It's a pity the 1.x era programmers have left us after going standalone
stgatilov

stgatilov

26.07.2021 04:00

administrator   ~0014185

Fixed Linux build in svn rev 9508.

I'm afraid I closed it too early.
It seems that the current way of specifying subtitles is too inconvenient for thousands of short sounds that we have.

Issue History

Date Modified Username Field Change
04.12.2010 09:50 tels New Issue
04.12.2010 09:50 tels Status new => assigned
04.12.2010 09:50 tels Assigned To => tels
04.03.2011 17:12 tels Assigned To tels =>
04.03.2011 17:12 tels Status assigned => new
19.07.2011 19:45 tels Relationship added related to 0002779
19.07.2011 19:45 tels Assigned To => tels
19.07.2011 19:45 tels Status new => assigned
19.07.2011 19:45 tels Product Version => TDM 1.06
19.07.2011 20:27 tels Relationship added has duplicate 0001858
19.06.2012 21:36 tels Assigned To tels =>
03.01.2015 16:14 grayman Status assigned => new
06.08.2018 14:24 nbohr1more Note Added: 0010739
15.05.2020 04:00 nbohr1more Assigned To => duzenko
15.05.2020 04:00 nbohr1more Status new => assigned
03.07.2021 18:34 duzenko Note Added: 0014146
03.07.2021 18:40 duzenko Note Edited: 0014146 View Revisions
14.07.2021 11:26 stgatilov Note Added: 0014153
25.07.2021 13:35 stgatilov Target Version => TDM 2.10
25.07.2021 13:56 stgatilov Note Added: 0014181
25.07.2021 14:16 stgatilov Note Added: 0014182
25.07.2021 14:25 stgatilov Note Added: 0014183
25.07.2021 14:26 stgatilov Note Edited: 0014183 View Revisions
25.07.2021 14:27 stgatilov Status assigned => resolved
25.07.2021 14:27 stgatilov Resolution open => fixed
25.07.2021 14:27 stgatilov Fixed in Version => TDM 2.10
25.07.2021 20:05 duzenko Note Added: 0014184
26.07.2021 04:00 stgatilov Note Added: 0014185
26.07.2021 04:00 stgatilov Status resolved => feedback
26.07.2021 04:00 stgatilov Resolution fixed => reopened
26.07.2021 04:00 stgatilov Status feedback => assigned