cChardet/readme.md

# cChardet
This library is high speed universal character encoding detector. - binding to [charsetdetect](https://bitbucket.org/medoc/uchardet-enhanced/overview).

This library is faster than [chardet](http://pypi.python.org/pypi/chardet).

# Support codecs
*   Big5
*   EUC-JP
*   EUC-KR
*   GB18030
*   gb18030
*   HZ-GB-2312
*   IBM855
*   IBM866
*   ISO-2022-CN
*   ISO-2022-JP
*   ISO-2022-KR
*   ISO-8859-2
*   ISO-8859-5
*   ISO-8859-7
*   ISO-8859-8
*   KOI8-R
*   Shift_JIS
*   TIS-620
*   UTF-8
*   UTF-16BE
*   UTF-16LE
*   UTF-32BE
*   UTF-32LE
*   windows-1250
*   windows-1251
*   windows-1252
*   windows-1253
*   windows-1255
*   x-euc-tw
*   X-ISO-10646-UCS-4-2143
*   X-ISO-10646-UCS-4-3412
*   x-mac-cyrillic

# Requires
*   Cython: [http://www.cython.org/](http://www.cython.org/)

*   uchardet-enhanced: [https://bitbucket.org/medoc/uchardet-enhanced/overview](https://bitbucket.org/medoc/uchardet-enhanced/overview)

# Install
### Build uchardet-enhanced
1.   $cd /tmp

2.   $hg clone https://bitbucket.org/medoc/uchardet-enhanced

3.   $cd uchardet-enhanced/libcharsetdetect

4.   $./configure

5.   $make

6.   $sudo make install

7.   $ls -la /usr/local/lib

8.   $ls -la /usr/local/include

### Build cChardet
1.   $cd /tmp

2.   $git clone git://github.com/PyYoshi/cChardet.git

3.   $cd cChardet

4.   $sudo pip install or easy_install -U cython. (If your os is Ubuntu, I recommend that you do "sudo apt-get install python-dev cython")

5.   $python setup.py build

6.   $sudo python setup.py install

# Example

```python
# coding: utf8
import cchardet
msg = file(r"testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt").read()
result = cchardet.detect(msg)
print(result)
```

# Test
*   $sudo easy_install or pip install -U chardet nose

*   $nosetests --nocapture tests.py

# Benchmark
see [tests.TestCchardetSpeed](https://github.com/PyYoshi/cChardet/blob/master/tests.py#L414)

### Sample(shift_jis):
*   [testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt](https://github.com/PyYoshi/cChardet/blob/master/testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt)

### PC Spec.:
*   CPU: Intel Core i7 860 2.8GHz

*   RAM: DDR3-1333 16GB

*   Platform: Windows 7 HP x64, Python 2.7.3 32-bit

### Result:
*   chardet: 4.009999990463257s, shift_jis

*   cchardet: 0.0009999275207519531s, shift_jis

# Contact
[My blog](http://blog.remu.biz)

Sorry for my poor English :)
first commit 2012-06-20 01:41:36 +00:00			`# cChardet`
updated 2012-06-20 15:07:12 +00:00			`This library is high speed universal character encoding detector. - binding to [charsetdetect](https://bitbucket.org/medoc/uchardet-enhanced/overview).`

			`This library is faster than [chardet](http://pypi.python.org/pypi/chardet).`

			`# Support codecs`
			`* Big5`
			`* EUC-JP`
			`* EUC-KR`
			`* GB18030`
			`* gb18030`
			`* HZ-GB-2312`
			`* IBM855`
			`* IBM866`
			`* ISO-2022-CN`
			`* ISO-2022-JP`
			`* ISO-2022-KR`
			`* ISO-8859-2`
			`* ISO-8859-5`
			`* ISO-8859-7`
			`* ISO-8859-8`
			`* KOI8-R`
			`* Shift_JIS`
			`* TIS-620`
			`* UTF-8`
			`* UTF-16BE`
			`* UTF-16LE`
			`* UTF-32BE`
			`* UTF-32LE`
			`* windows-1250`
			`* windows-1251`
			`* windows-1252`
			`* windows-1253`
			`* windows-1255`
			`* x-euc-tw`
			`* X-ISO-10646-UCS-4-2143`
			`* X-ISO-10646-UCS-4-3412`
			`* x-mac-cyrillic`
first commit 2012-06-20 01:41:36 +00:00
			`# Requires`
updated 2012-06-20 15:07:12 +00:00			`* Cython: [http://www.cython.org/](http://www.cython.org/)`
first commit 2012-06-20 01:41:36 +00:00
updated 2012-06-20 15:07:12 +00:00			`* uchardet-enhanced: [https://bitbucket.org/medoc/uchardet-enhanced/overview](https://bitbucket.org/medoc/uchardet-enhanced/overview)`
first commit 2012-06-20 01:41:36 +00:00
wrote Install, etc... 2012-06-20 13:18:38 +00:00			`# Install`
			`### Build uchardet-enhanced`
updated 2012-06-20 15:07:12 +00:00			`1. $cd /tmp`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`2. $hg clone https://bitbucket.org/medoc/uchardet-enhanced`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`3. $cd uchardet-enhanced/libcharsetdetect`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`4. $./configure`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`5. $make`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`6. $sudo make install`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`7. $ls -la /usr/local/lib`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`8. $ls -la /usr/local/include`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
			`### Build cChardet`
updated 2012-06-20 15:07:12 +00:00			`1. $cd /tmp`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`2. $git clone git://github.com/PyYoshi/cChardet.git`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`3. $cd cChardet`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`4. $sudo pip install or easy_install -U cython. (If your os is Ubuntu, I recommend that you do "sudo apt-get install python-dev cython")`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`5. $python setup.py build`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
updated 2012-06-20 15:07:12 +00:00			`6. $sudo python setup.py install`

			`# Example`

			```python
			`# coding: utf8`
			`import cchardet`
updated 2012-06-20 15:11:47 +00:00			`msg = file(r"testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt").read()`
			`result = cchardet.detect(msg)`
updated 2012-06-20 15:07:12 +00:00			`print(result)`
			```

			`# Test`
fix 2012-06-20 15:15:49 +00:00			`* $sudo easy_install or pip install -U chardet nose`
updated 2012-06-20 15:07:12 +00:00
			`* $nosetests --nocapture tests.py`
first commit 2012-06-20 01:41:36 +00:00
add benchmark 2012-06-20 02:29:50 +00:00			`# Benchmark`
updated 2012-06-20 15:07:12 +00:00			`see [tests.TestCchardetSpeed](https://github.com/PyYoshi/cChardet/blob/master/tests.py#L414)`
fix 2012-06-20 02:31:41 +00:00
add spec 2012-06-20 02:40:03 +00:00			`### Sample(shift_jis):`
updated 2012-06-20 15:07:12 +00:00			`* [testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt](https://github.com/PyYoshi/cChardet/blob/master/testdata/wikipediaJa_One_Thousand_and_One_Nights_SJIS.txt)`
fix 2012-06-20 02:31:41 +00:00
add spec 2012-06-20 02:40:03 +00:00			`### PC Spec.:`
updated 2012-06-20 15:07:12 +00:00			`* CPU: Intel Core i7 860 2.8GHz`
fix 2012-06-20 02:31:41 +00:00
updated 2012-06-20 15:07:12 +00:00			`* RAM: DDR3-1333 16GB`
add spec 2012-06-20 02:40:03 +00:00
updated 2012-06-20 15:07:12 +00:00			`* Platform: Windows 7 HP x64, Python 2.7.3 32-bit`
wrote Install, etc... 2012-06-20 13:18:38 +00:00
add spec 2012-06-20 02:40:03 +00:00			`### Result:`
updated 2012-06-20 15:07:12 +00:00			`* chardet: 4.009999990463257s, shift_jis`
fix 2012-06-20 02:31:41 +00:00
updated 2012-06-20 15:07:12 +00:00			`* cchardet: 0.0009999275207519531s, shift_jis`
add benchmark 2012-06-20 02:29:50 +00:00
first commit 2012-06-20 01:41:36 +00:00			`# Contact`
			`[My blog](http://blog.remu.biz)`

			`Sorry for my poor English :)`