1、新增中文编码
|
1 2 3 4 5 |
sudo vim /var/lib/locales/supported.d/local # 添加 zh_CN.UTF-8 UTF-8 en_US.UTF-8 UTF-8 zh_CN.GB18030 GB18030 |
2、生成编码
|
1 |
sudo locale-gen |
3、设置默认的编码
|
1 2 3 |
sudo vi /etc/environment # 添加 LANG="en_US.UTF-8" |
重新登录shell后，即可生效。
更新：如果Ubuntu 16.04后找不到第一个文件了，可以用/etc/locale.gen这个文件
1、报错：
|
1 2 3 |
import urllib2 url = u"http://www.baidu.com/wd=测试" urllib2.urlopen(url).read() |
错误如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 404, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 422, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 1214, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open h.request(req.get_method(), req.get_selector(), req.data, headers) File "/usr/lib/python2.7/httplib.py", line 973, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 969, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 829, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 805, in send self.sock.sendall(data) File "/usr/lib/python2.7/socket.py", line 224, in meth return getattr(self._sock,name)(*args) UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-9: ordinal not in range(128) |
2、解决方法
|
1 2 3 |
import urllib2 url = u"http://www.baidu.com/wd=测试" urllib2.urlopen(url.encode('utf-8')).read() |
很简单，对url进行encode即可：
转自：http://www.cnblogs.com/hopeworld/archive/2011/04/20/2022331.html
Vim有四个跟字符编码方式有关的选项，encoding、fileencoding、fileencodings、termencoding(这些选项设置请参考Vim文档中encoding-names章节)，它们的意义如下:
|
1 2 3 4 5 6 7 8 |
import urllib2 from BeautifulSoup import BeautifulSoup page = urllib2.urlopen('http://www.leeon.me'); soup = BeautifulSoup(page,fromEncoding="gb18030") print soup.originalEncoding print soup.prettify() |
如果中文页面编码是gb2312，gbk，在BeautifulSoup构造器中传入fromEncoding=”gb18030″参数即可解决乱码问题，即使分析的页面是utf8的页面使用gb18030也不会出现乱码问题！
2012.9.19更新：
实际上，fromEncoding=”gb18030″并不是一劳永逸的方法，当面对iso-8859-1编码的中文网页时[……]
首先，mutt是很智能的，不要低估了人家的智商。
默认来说，是根据系统的LANG变量来评估所需要试用的编码的。
以中文为例
|
1 2 3 4 5 6 7 8 9 10 11 12 |
sudo sudo dpkp-reconfigure locales #然后构上选择zh_CN.GB2312 #但是不要选择默认编码！ Generating locales (this might take a while)... zh_CN.GB2312... done Generation complete. #执行成功后直接export export LANG=zh_CN.GB2312 好了，现在是彻底支持中文的了。 |