Results 1 to 11 of 11

Thread: Emoji Encoding

  1. #1

    Default Emoji Encoding

    Something I'm ready to complain about: many browser (Chrome, IE, but displays on Mozilla) forms of emoji do not render on that page, or when pasted* in text. For example:

    ��
    ��
    ��
    ��

    The middle two are not rendered anywhere for me. This is especially noticeable on Youtube, where comments look full of the (non-represented) rectangles.
    What do?

    EDIT: While some displayed in the Org text prompt, none of them do in the post.


    *How do you type these unicodes? I can never seem to get it.
    Last edited by Montmorency; 01-12-2019 at 05:25.
    Vitiate Man.

    History repeats the old conceits
    The glib replies, the same defeats


    Spoiler Alert, click show to read: 



  2. #2
    Iron Fist Senior Member Husar's Avatar
    Join Date
    Jan 2003
    Location
    Germany
    Posts
    15,617

    Default Re: Emoji Encoding

    I'm not entirely sure about the science, but I think you just don't.
    They will mostly work in personal correspondence such as messengers and email, but not on most webpages.
    On webpages, you usually have to go with the set provided by the webpage or hope that the other person is using a (compatible) phone themselves. Some phone manufacturers seem to offer emojis that don't work on other manufacturers' phones and the implementation can obviously differ as well. I can't say that I ever noticed working smileys in youtube comments and I read them quite frequently.

    If I'm not mistaken, that is because the browser web engines usually do not contain any emojis and the parsing of the text (that replaces the signs with the picture of the emoji) is done by the webserver itself and then sent to the browser in completed form. The browser and the operating system will then not really replace anything anymore (outside of the replacement instructions as sent by the webserver, which do not include unicode emojis). There used to be some exceptions IIRC, but that was long ago.

    It's different in apps and the like, but apps like whatsapp don't use the smileys of the operating system and have their own set instead. Windows 10 also has emojis, but I doubt they will work here for said reasons. They can be found either in the on-screen keyboard or by using a special key combination (Windows key + Ü in Germany, obviously different elsewhere, I think windows key + . ).

    That's my take on it, if anyone knows how this mess actually works, I'm interested as well.

    ����‍������ <- these actually work for me, who else can see them?....aaaaand after editing the post to add that they work, they're gone at least that saves my theory above
    Last edited by Husar; 01-12-2019 at 13:57.


    "Topic is tired and needs a nap." - Tosa Inu

  3. #3
    Senior Member Senior Member Yeti Sports 1.5 Champion, Snowboard Slalom Champion, Monkey Jump Champion, Mosquito Kill Champion Csargo's Avatar
    Join Date
    May 2005
    Location
    Vote:Sasaki
    Posts
    13,331

    Default Re: Emoji Encoding

    From googling this is apparently an issue with fonts not having the appropriate Unicode support for the vast array of different symbols associated with them. So whatever font your phone, PC, and I'm guessing that extends to webpages as well, are incapable of encompassing all of the Unicode symbols/emoji's that exist. From what I read it's not possible to have a font that's capable of displaying every Unicode symbol, because they can't fit them all into one font, and the number of symbols continues to grow. I'm not sure how it relates to webpages and browsers. I'm guessing the webpage would supersede the browser when displaying stuff on the page, but I'm not sure if that's true. I.e. it wouldn't matter what fonts you were using in the browser if a webpage had it's own specific font it was using, so your browser would potentially be able to display the Unicode, but the webpage wouldn't. I don't quite know how it all works though, so this could be completely wrong.
    Quote Originally Posted by Sooh View Post
    I wonder if I can make Csargo cry harder by doing everyone but his ISO.

  4. #4
    Iron Fist Senior Member Husar's Avatar
    Join Date
    Jan 2003
    Location
    Germany
    Posts
    15,617

    Default Re: Emoji Encoding

    If it's the fonts, then I wonder how Smartphones do it since they can also mix letters and symbols. Same for email or even MS Word. I just tried it and even Excel displays very basic black and white emojis when I enter them there. It's almost only on webpages/internet forums that you run into display issues. Or when two smartphones/messenger versions have incompatible sets. E.g. when a friend had a version of WhatsApp with more smileys than my version (Android vs WindowsPhone), I couldn't see those additional ones.

    Obviously in an app there could be a special implementation, like here in the forums, but pretty much every modern OS implements smileys in a native way, which is why they work in Word, Excel etc. The question would be why those aren't shown in the forum here? My guess is that the OS is not parsing the text shown in the browser, so the OS implementation does not matter. They do work upon entering, which is because the code that I enter is executed on my computer (local JavaScript I guess), but once I send the post to the server, they are gone, because the server cannot handle that code.

    Example: ������


    "Topic is tired and needs a nap." - Tosa Inu

  5. #5
    Iron Fist Senior Member Husar's Avatar
    Join Date
    Jan 2003
    Location
    Germany
    Posts
    15,617

    Default Re: Emoji Encoding

    Now the screenshots, maybe @therother can explain the exact details.

    Click image for larger version. 

Name:	emojis.jpg 
Views:	177 
Size:	424.2 KB 
ID:	22058

    Remember that before I post, the text is only on my computer and after I posted, the browser displays the text that the server sends to my browser. Apparently the emojis are lost when the server creates the post for displaying in the browser. It could still be about fonts, but then there seems to be a fundamental difference in how fonts are handled on websites and in apps, etc.
    Last edited by Husar; 01-12-2019 at 21:17.


    "Topic is tired and needs a nap." - Tosa Inu

  6. #6

    Default Re: Emoji Encoding

    Good stuff, but as I noted - for Youtube at least - the emojis display in Firefox but not Chrome or IE.

    And clearly a lot of people can see and use them, because they show up frequently enough.

    Still can't figure out key commands. For example, how do I type out U+1F600? These instructions don't cut it (not that there's much practical reason to be typing out emojis):


    Inserting Unicode characters
    To insert a Unicode character, type the character code, press ALT, and then press X. For example, to type a dollar symbol ($), type 0024, press ALT, and then press X. For more Unicode character codes, see Unicode character code charts by script.

    Important: Some of the Microsoft Office programs, such as PowerPoint and InfoPath, cannot convert Unicode codes to characters. If you need a Unicode character and are using one of the programs that doesn't support Unicode characters, use the Character Map to enter the character(s) that you need.

    Notes:

    If ALT+X converts the wrong character code into Unicode, select the correct character code before pressing ALT+X.

    Alternatively, precede the correct character code with the text "U+". For example, typing "1U+B5" and pressing ALT+X will always return the text "1µ", while typing "1B5" and pressing ALT+X will return the text
    Vitiate Man.

    History repeats the old conceits
    The glib replies, the same defeats


    Spoiler Alert, click show to read: 



  7. #7
    Iron Fist Senior Member Husar's Avatar
    Join Date
    Jan 2003
    Location
    Germany
    Posts
    15,617

    Default Re: Emoji Encoding

    Quote Originally Posted by Montmorency View Post
    Good stuff, but as I noted - for Youtube at least - the emojis display in Firefox but not Chrome or IE.

    And clearly a lot of people can see and use them, because they show up frequently enough.

    Still can't figure out key commands. For example, how do I type out U+1F600? These instructions don't cut it (not that there's much practical reason to be typing out emojis):
    You're right, I forgot that you mentioned the browsers (shame on me). They do show up for me on Youtube, in the app and in Edge on my phone and also in my Vivaldi browser, which is based on Chromium. Edge on the desktop has them as well.
    They still don't show up here on the Org though. Which makes it really strange because now it seems like in some cases it depends on the website and in others on the browser.

    What is your OS? Do you use Windows 10 or an older one? Since you mentioned IE but not Edge. There could be an element of OS support as well.


    "Topic is tired and needs a nap." - Tosa Inu

  8. #8
    Research Fiend Technical Administrator Tetris Champion, Summer Games Champion, Snakeman Champion, Ms Pacman Champion therother's Avatar
    Join Date
    Feb 2004
    Location
    UK
    Posts
    2,631

    Default Re: Emoji Encoding

    I'm afraid the more exotic characters have always been a black box for me as well.

    As to why some won't work on the Org, I suspect this is due to the database using the (then) default MySQL database collation latin1, which is a single byte character set. latin1 doesn't support as wide an array of characters that a multibyte character set such as utf8, which is I believe the current default. We moved to vBulletin 15 years ago, which was when the decision was made. IIRC, converting the database to utf8 isn't straightforward, particularly such a large and old database as the Org's.
    Nullius addictus iurare in uerba magistri -- Quintus Horatius Flaccus

    History is a pack of lies about events that never happened told by people who weren't there -- George Santayana

    Members thankful for this post (2):



  9. #9

    Default Re: Emoji Encoding

    Quote Originally Posted by Husar View Post
    You're right, I forgot that you mentioned the browsers (shame on me). They do show up for me on Youtube, in the app and in Edge on my phone and also in my Vivaldi browser, which is based on Chromium. Edge on the desktop has them as well.
    They still don't show up here on the Org though. Which makes it really strange because now it seems like in some cases it depends on the website and in others on the browser.

    What is your OS? Do you use Windows 10 or an older one? Since you mentioned IE but not Edge. There could be an element of OS support as well.
    W7

    Quote Originally Posted by therother View Post
    I'm afraid the more exotic characters have always been a black box for me as well.

    As to why some won't work on the Org, I suspect this is due to the database using the (then) default MySQL database collation latin1, which is a single byte character set. latin1 doesn't support as wide an array of characters that a multibyte character set such as utf8, which is I believe the current default. We moved to vBulletin 15 years ago, which was when the decision was made. IIRC, converting the database to utf8 isn't straightforward, particularly such a large and old database as the Org's.
    Raises the specter of the Org shutting down as maintenance becomes unmanageable with time.
    Vitiate Man.

    History repeats the old conceits
    The glib replies, the same defeats


    Spoiler Alert, click show to read: 



  10. #10
    Iron Fist Senior Member Husar's Avatar
    Join Date
    Jan 2003
    Location
    Germany
    Posts
    15,617

    Default Re: Emoji Encoding

    Quote Originally Posted by Montmorency View Post
    W7
    That could easily be part of your problem. Perhaps Firefox has some kind of built-in emoji support while IE and Chrome rely on the OS for that. I don't think W7 has native emoji support like Win10 does. Does not seem like the sort of thing they would patch in shortly before the end of official support.

    edit with further research: https://blog.getemoji.com/post/57054...-support-emoji

    Windows — Windows 8 and 8.1

    Internet Explorer — Yes!
    Firefox — Yes!
    Safari — Yes!
    Chrome — No ☹
    ☹ If using Windows 7, Windows Vista, or any other early version of Windows, you will not see Emoji correctly in any browser.
    And to finally nail the Firefox thing as well: https://www.mozilla.org/en-US/firefo.../releasenotes/

    Added a built-in Emoji set for operating systems without native Emoji fonts (Windows 8.0 and lower and Linux)
    So it's basically that your Windows does not support emojis, but Firefox has special built-in emoji support since version 50. That's why you only see them in Firefox and nowhere else.

    You could have a look at this, but not sure what exactly could go wrong or what it does: https://answers.microsoft.com/en-us/...d-24b7bec8c444
    Last edited by Husar; 01-14-2019 at 01:34.


    "Topic is tired and needs a nap." - Tosa Inu

    Member thankful for this post:



  11. #11

    Default Re: Emoji Encoding

    great work

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Single Sign On provided by vBSSO