What is necessary for the next input method framework?

2018/08/11 1/23
What is necessary for
the next input method framework
Fuminobu TAKEYAMA
(ftake)
openSUSE.Asia Summit 2018, Taipei

2018/08/11 2/23
About me
●
TAKEYAMA, Fuminobu （武山文信）
– openSUSE ID: ftake
– from Japan
●
Community (weekend) developer
– Maintainer of openSUSE M17N (Multilingualization)
●
opensuse-m17n@opensuse.org
●
https://build.opensuse.org/projects/M17N
– Not a developer of any specific input method framework
●
Communities
– Organizer of Japan openSUSE User Group
– openSUSE.Asia Summit 2017 Local Organizing Chair

2018/08/11 3/23
What is an input method?
●
A mechanism to input various characters
– Complex characters used in Chinese, Japanese, Korean
– Usually using several key strokes
aoi
あおい
青い

2018/08/11 4/23
What is an input method framework
(IMF) ?
●
An interface between applications and
input method engines (conversion software)
– Also allows switching multiple input method engines
– e.g., IBus, Fcitx, GCIN, SCIM, UIM, IIIMF, Kinput2, ...
GTK App. Qt App. X App.
IMF daemon
*-qt *-xim*-gtk IMFGUI
QtGtk X server
Mozc
libchewing3
*-mozc
*-chewing
Engines
Different API for each GUI platform
GTK, Qt IM module, XIM
An unified API to
implement engines

2018/08/11 5/23
Input method frameworks on
openSUSE
●
openSUSE supports multiple IMFs
●
Different IMF is used depending on the session locale by
default
– GCIN for zh_TW, zh_HK
– Fcitx for zh_ CN, etc.
– IBus for ja_JP

2018/08/11 6/23
Recent changes in the API layer
●
Qt5 contains ibus-qt (GTK+ 4 will also)
●
Flatpak applications can communicate with IBus running on
the host
– using the IBus portal D-Bus interface
GTK App. Qt App. X App.
ibus-daemon
ibus-qt ibus-ximibus-gtk
QtGtk X server
Mozc
libchewing3
ibus-mozc
ibus-chewing
IBus D-Bus Interface
Flatpak App.
Flatpak portal
Engines
...
...
ibus-ui-gtk3ibus-ui-gtk3

2018/08/11 7/23
But, Flatpak supports only IBus
●
Issue #43 Add other im module for input method
– https://github.com/flatpak/freedesktop-sdk-images/issues/4
3
I'm still not convinced we want to support multiple input
method frameworks
By Matthias Clasen

2018/08/11 8/23
Some people might think
IBus is now the standard IMF?
●
No
– Quite many people prefer Fcitx or Gcin and hate IBus
– By Googling “ibus”, several blog posts saying “how ibus is
bad” are found
– Due to the several design issues (mentioned later)
●
Why so many IMFs have been developed?
– Formerly, for more stability
●
Let’s remind conversion engines were running on an application
process
– Now, usability, customizability, ...

2018/08/11 9/23
Do we still need various IMFs?
●
I think no if we have a thin and simple IMF
– No major difference in the core part of today’s IMFs
– “An interface between applications and engines” does not
provide much user experience
– GUI should be separated into engines and replaceable
●
No more multiple IMF support cost
●
It’s time to end the war of input method framework

2018/08/11 10/23
So let’s discuss
what is necessary for the next IMF?
●
From the viewpoint of architecture/design
●
By reviewing the problems of IBus (and Fcitx)

2018/08/11 11/23
1. Separation of GUI from core IMF
service 1/2
●
GUI of input method is now provided by IMF
– Design of the GUI is a major cause of discontent
●
A candidate window should be a part of an engine
– e.g., IBus cannot show hints (word usage) on a candidate list
●
Mozc replaces candidate window with its own in irregular way
but it is broken on GNOME Wayland shell
Candidate window
of ibus-ui-gtk3
(with ibus-kkc)
Candidate window of Mozc

2018/08/11 12/23
1. Separation of GUI from core IMF
service 2/2
●
For IBus, some major logic of IMF is implemented in
GUI layer (status panel applications)
– Switching keyboard layout, loading plugins, etc.
●
What is worse, there are three GUI layer variations
– ibus-ui-gtk3
– GNOME keyboard plugin
– KDE Input method panel

2018/08/11 13/23
2. Running without specific GUI
libraries
●
IBus depends on GTK+
– KDE input method panel still requires ibus-ui-gtk3
– Fcitx have less dependency
●
Will be necessary for embedded
– e.g., Automotive Grade Linux

2018/08/11 14/23
3. Wayland support
●
Need APIs between IM engine and Wayland compositor
(server)
– To get absolute position of text cursor
– To place IM engine’s window to appropriate position

2018/08/11 15/23
4. Redesigned keyboard layout
switch 1/4
●
Both IBus and Fcitx support this for layouts such as
Russian and Arabic
– ASDF ⇔ ФЫВА
By Nkoke, No modification, CC-BY-SA 3.0
https://commons.wikimedia.org/wiki/File:Russian_keyboard_win.png
A S D F

2018/08/11 16/23
switch 2/4
●
A layout is provided as an IBus engine
– ibus-simple
– Some engine like Anthy allow to specify a layout too

2018/08/11 17/23
switch 3/4
●
The following situations are not supported
– To switch between JP and US layout with engines like Anthy,
Mozc...
– To use two keyboards with different layout (US, JP)
Internal JP layout
USB US layout

2018/08/11 18/23
switch 4/4
●
Switching pairs of an engine and a layout is necessary
●
Anthy JP
●
Anthy US
●
Chewing US
●
Simple RU
– The same engine can be registered multiple times
●
Hardware-specific layout settings
– Internal Keyboard: JP
– USB Keyboard: US

2018/08/11 19/23
5. Support of text input without
physical keyboard
●
Integration with input and conversion
is necessary
– Should be implemented
as an “engine” (plugin)
●
Screen keyboard
– Integrated conversion UI
– Need API to show up
●
Voice input
– Converted texts are directly passed to
applications ATOK input method on Android

2018/08/11 20/23
6. Input mode v.s. input engine 1/2
●
IBus 1.5 is designed for CJK users to use two engines
by switching Meta+Space
– Conversion engine: Anthy, Mozc, Chewing, Pinyin
– ibus-simple: for direct input (a-z, 0-9, ...)
●
Problems
– 半角/全角 key is not used to turn on/off
●
Even though most Japanese use
●
IBus need modifier keys to show its engine selector window
while the keys are pressed
– No shortcut to switch to direct input mode
Turning of/off IBus was obsoleted

2018/08/11 21/23
6. Input mode v.s. input engine 2/2
●
Workaround in IBus 1.5 for CJK
– Use local mode of each engine and do not use ibus-simple
(shortcut keys are provided by engines)
●
“A” mode and “あ” mode in Anthy, Mozc
●
“英” mode and “中” mode in Chewing
●
What should this be?
– Exposing each local mode as an engine like macOS X
(Not sure this is the best)
●
Direct (Simple) RU
●
“A” (Mozc) US
●
“あ” (Mozc) US
●
“中” (Chewing) US Meta+Space
Settings for Russian,
Japanese and Traditional Chinese

2018/08/11 22/23
7. Open community
●
It is getting difficult to make a good solution without
open discussion
– An input method framework is involved with many parts of
applications and platform
●
Not only GNOME
– If the design changes in IBus 1.5 was discussed openly, we
could have resolved some problems before the release

2018/08/11 23/23
Conclusion
●
I think it is possible to make a standard IMF
– By separating into thin core service and various engine plugins
●
Let’s use our effort not for multiple IMF support but for
– New input mechanism such as screen keyboard and voice input
– Wayland integration
– Resolving application specific issues
●
Cursor position problem
●
Electron applications bug in Flatpak

What is necessary for the next input method framework?

Recommended

Recommended

More Related Content

Similar to What is necessary for the next input method framework?

Similar to What is necessary for the next input method framework? (20)

More from Fuminobu Takeyama

More from Fuminobu Takeyama (20)

Recently uploaded

Recently uploaded (20)

What is necessary for the next input method framework?