<commit_before>---
layout: post
title: "Python: Generating pdf containing emojis using reportlab"
summary: Creating pdf docs that supports emojis. We use "Emojipy" python library to parse emoji's unicode
comments: True
twitter: True
tags: [python, emoji, reportlab]
---

## Understanding Emoji
[Emojis](https://en.wikipedia.org/wiki/Emoji) 😃 🎉 👍 💕 have become so popular. Anyone who used [Whatsapp](https://www.whatsapp.com/) or
[Instagram](https://instagram.com/) knows how cool emojis are 😎. And they are everywhere now. Emoji originated in Japan. The word "emoji" means "Picture(e) letter(moji)" in Japanese.

Technically, emojis are [subset](http://unicode.org/emoji/charts/full-emoji-list.html) of [Unicode character set](http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt).  In short, Unicode is the collection of every writable symbol available on the planet. That being said unicode includes symbols from every language that exist.

There are different types of encoding used to interpret different sets of unicode. Most popular is "UTF-8" encoding. To understand how unicode are
interpreted, stored and retrieved here is a great article [The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About
Unicode and Character Sets (No Excuses!)](http://www.joelonsoftware.com/articles/Unicode.html)

Every emoji is a single unicode character that can be upto 4 bytes long depending on type of encoding used. <strong>Yes, every character is just one byte is no longer true.</strong>

<!--break-->

### Interpreting Emoji
The letter <strong>'A'</strong> corresponds to <strong>'65'</strong> in decimal and <strong>'0x41'</strong> in hex. Similarly emoji 😃 corresponds
to <strong>'128515'</strong> in decimal and <strong>'0x1F603'</strong> in hexadecimal. Though every emoji(unicode) can be represented in decimal or
hexadecimal, the standard way to denote emoji is by 'code point'.

code point for is 😃 is <strong>U+1F603</strong>. Every emoji(unicode) is represented in code point as "U+" (to denote unicode) and followed by its
hex value "1F603". In Python it is represented as ```"\U0001f603" (u"\U0001f603" in python 2)```.
[Here](http://apps.timwhitlock.info/emoji/tables/unicode) is a list of emojis with its code points.

Every emoji have a name. Some apps like Github, Slack, Basecamp, etc, provide short name for emoji to make it easy to remember. You can find the list
of emoji with its short name here [emoji-cheat-sheet](http://www.emoji-cheat-sheet.com/)

### Interpreting with Python
Get the name and category of an emoji using ```unicodedata``` module of python
{% highlight python %}<commit_msg>some typos<commit_after>---
layout: post
title: "Python: Generating pdf containing emojis using reportlab"
summary: Creating pdf docs that supports emojis. We use "Emojipy" python library to parse emoji's unicode
comments: True
twitter: True
tags: [python, emoji, reportlab]
---

## Understanding Emoji
[Emojis](https://en.wikipedia.org/wiki/Emoji) 😃 🎉 👍 💕 have become so popular. Anyone who used [Whatsapp](https://www.whatsapp.com/) or
[Instagram](https://instagram.com/) knows how cool emojis are 😎. And they are everywhere now. Emoji originated in Japan. The word "emoji" means "Picture(e) letter(moji)" in Japanese.

Technically, emojis are [subset](http://unicode.org/emoji/charts/full-emoji-list.html) of [Unicode character set](http://www.unicode.org/Public/UCD/latest/ucd/UnicodeData.txt).  In short, Unicode is the collection of every writable symbol available on the planet. That being said, unicode includes symbols from every language that exist.

There are different types of encoding used to interpret different sets of unicode. Most popular is "UTF-8" encoding. To understand how unicode are
interpreted, stored and retrieved here is a great article [The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About
Unicode and Character Sets (No Excuses!)](http://www.joelonsoftware.com/articles/Unicode.html)

Every emoji is a single unicode character that can be upto 4 bytes long depending on type of encoding used. <strong>Yes, every character is just one byte is no longer true.</strong>

<!--break-->

### Interpreting Emoji
The letter <strong>'A'</strong> corresponds to <strong>'65'</strong> in decimal and <strong>'0x41'</strong> in hex. Similarly emoji 😃 corresponds
to <strong>'128515'</strong> in decimal and <strong>'0x1F603'</strong> in hexadecimal. Though every emoji(unicode) can be represented in decimal or
hexadecimal, the standard way to denote emoji is by 'code point'.

Code point for 😃 is <strong>U+1F603</strong>. An emoji(unicode) in code point is represented as "U+" (to denote unicode) followed by its
hex value "1F603". In Python it is represented as ```"\U0001f603" (u"\U0001f603" in python 2)```.
[Here](http://apps.timwhitlock.info/emoji/tables/unicode) is a list of emojis with its code points.

Every emoji have a name. Some apps like Github, Slack, Basecamp, etc, provide short name for emoji to make it easy to remember. You can find the list
of emoji with its short name here [emoji-cheat-sheet](http://www.emoji-cheat-sheet.com/)

### Interpreting with Python
Get the name and category of an emoji using ```unicodedata``` module of python
{% highlight python %}
{'fraction_non_alphanumeric': 0.1035592578568724, 'fraction_numerical': 0.01552442256720939, 'mean_word_length': 4.87, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 6, 'lorem ipsum': 0, 'www.': 8, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5829311', 'n_tokens_mistral': 1632, 'n_tokens_neox': 1551, 'n_words': 671}