前言
本不想多说什么,年后在公司亲历一段别样经验,不想吐槽,只因吐槽不能改变任何现状。小姐心态,寡妇待遇,妇联追求,一份技术工作做出弯腰,低头,下跪,也是醉了。今年大环境不好,大厂裁员,人才过盛,好在自己还是去了符合自己意愿的公司。
新工作快一个月,主要工作技术内容是一个Django的小东西,首先是需要从Python2 迁移至Python3,简单带点重构。
Python2 迁移Python3 运行环境
服务run在docker里,启动方式里面没有嵌入太多环境变量,基本基础环境。但是Django自身依赖一些库环境需要解决。
pycurl
安装 pycurl可能会遇到 ssl lib的Error,建议执行:
export
PYCURL_SSL_LIBRARY
=
openssl
export
LDFLAGS
=
-L/usr/local/opt/openssl/lib
export
CPPFLAGS
=
-I/usr/local/opt/openssl/include
# 执行上述环境变量设之后,再安装
pip
install
pycurl
mysqlclient
mysqlclient比较坑,安装时候会抛出:
library not found for -lssl
running build_ext
building 'MySQLdb._mysql' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/MySQLdb
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -Dversion_info=(1,4,2,'post',1) -D__version__=1.4.2.post1 -I/usr/local/opt/mysql@5.7/include/mysql -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c MySQLdb/_mysql.c -o build/temp.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.o
gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.o -L/usr/local/opt/mysql@5.7/lib -lmysqlclient -lssl -lcrypto -o build/lib.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.cpython-37m-darwin.so
ld: library not found for -lssl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1
mysqlclient 官网上给出了比较清晰的安装方式https://github.com/PyMySQL/mysqlclient-python
debian
centos和debian下还比较好解决,直接系统安装 dev包即可
# 先安装
apt-get
install
python-dev default-libmysqlclient-dev
# 再安装mysqlclient
pip
install
mysqlclient
MacOS
MacOS下问题比较多,官网上给的很清楚,mysql-connector-c有bug
# 首先安装mysql
brew install mysql@
5.7
# 安装 mysql-connector-c
brew install mysql
-
connector
-
c
编辑/usr/local/bin/mysql_config 确保:
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
# 修改/usr/local/bin/mysql_config
vim /usr/local/bin/mysql_config
# 设置openssl环境变量
export
PATH
=
"/usr/local/opt/openssl/bin:
$PATH
"
export
LDFLAGS
=
"-L/usr/local/opt/openssl/lib"
export
CPPFLAGS
=
"-I/usr/local/opt/openssl/include"
# 再安装mysqlclient
pip
install
mysqlclient
基本环境设置好,直接安装依赖即可
Django
貌似最初开始接触Python就是Django,从Python2至Python3也不是一个容易的过程
Django 版本差异
1,python2.x下,Django最多支持版本应该是1.11,目前Django 都2.2了,中间相差太多
Python3下建议使用高版本
2,djangorestframework是Django用来做纯RESTFull API的,官网介绍如下:
https://www.django-rest-framework.org/#quickstart
并且新版本的设计使用方式,偏向于router register viewset形式,配合action:
https://www.django-rest-framework.org/api-guide/viewsets/
注意
1,低版本djangorestframework不支持 action写法,建议使用最新版本
2,router 写法需要严格村从RESTFull形式,router.register prefix中不支持 'pre/fix'形式,action中url_path也不支持 '/'和'.'
APPEND_SLASH
Django 在路由传统路由匹配时,会有结尾 ‘/’的区分,router.register和action中都是默认以‘/’结尾,
urlpatterns
=
[
url
(
r
'^user/$'
,
views
.
NodeView
.
as_view
(
)
)
,
# url(r'^user$', views.NodeView.as_view()),
]
以上两种实质是不一样的uri,但是Django默认APPEND_SLASH=True的,当:
MIDDLEWARE
=
[
.
.
.
'django.middleware.common.CommonMiddleware'
,
.
.
.
]
middleware 中开启CommonMiddleware,会对用户访问 http://doamin/user 进行自动redirect 至 http://doamin/user/
迁移过程中,很多url不规范,都是
r'^user$'
形式,整合到router.register形式就会有重定向问题,不仅如此,在主urls.py文件中:
urlpatterns
=
[
.
.
.
url
(
r
'.*'
,
views
.
main_view
)
,
# 没收所有没有匹配上的
]
@login_required
def
main_view
(
request
)
:
return
TemplateResponse
(
request
,
'index.html'
)
这种形式构造,原本 http://doamin/user 完全可以urlpatterns中未匹配任何,之后缺少 结尾’/'后重定向匹配http://doamin/user/,
现在被
r'.*'
没收,返回index界面,而且还需要是login的,否则跳转到login界面
proxy
django-proxy库用来对一个request进行proxy转发,在多数整合系统中用来做一个简单gateway很是便捷,Django自带的Redirect是
本域内的重定向,并非转发。https://github.com/mjumbewu/django-proxy,用法也很清晰。
关闭APPEND_SLASH=False,对上面自动重定向问题进行手动处理,自动匹配所有user prefix下的所有未具有’/'请求:
from
proxy
.
views
import
proxy_view
def
main_view
(
request
)
:
path
,
host
=
request
.
path
,
request
.
get_host
(
)
if
path
.
startswith
(
'/user/'
)
and
not
path
.
endswith
(
'/'
)
:
return
proxy_view
(
request
,
f
'http://{host}{path}/'
)
@login_required
def
logined_view
(
request
)
:
return
TemplateResponse
(
request
,
'index.html'
)
原有
r'.*'
在urlpatterns最末尾,访问http://doamin/user/直接就匹配上,访问http://doamin/user被
r'.*'
匹配,而后对
request中的path进行判定,如果是/user/起始前缀的,并且没有’/‘结尾,补充’/'后重新访问。
注意
此处用proxy,并非redirect,因为关于APPEND_SLASH的说明中给出,对于APPEND_SLASH会丢失 POST访问,所以此处用proxy一并解决
Django User
默认Django在 request 中获取user,即为获取User model中对应的访问user
def
response_func
(
request
)
:
print
(
request
.
user
)
背后跟session关联的,关于Django session的设定,Django session 官网给了比较详细的介绍
MIDDLEWARE
=
[
.
.
.
'django.contrib.sessions.middleware.SessionMiddleware'
,
.
.
.
'django.contrib.auth.middleware.AuthenticationMiddleware'
,
.
.
.
]
中通过利用session确定用户。
class
SessionMiddleware
(
MiddlewareMixin
)
:
def
__init__
(
self
,
get_response
=
None
)
:
self
.
get_response
=
get_response
engine
=
import_module
(
settings
.
SESSION_ENGINE
)
self
.
SessionStore
=
engine
.
SessionStore
def
process_request
(
self
,
request
)
:
session_key
=
request
.
COOKIES
.
get
(
settings
.
SESSION_COOKIE_NAME
)
request
.
session
=
self
.
SessionStore
(
session_key
)
settings.SESSION_ENGINE 设置了session的存储引擎,默认
django.contrib.sessions.backends.db
就是用db
settings.SESSION_COOKIE_NAME默认的名称
sessionid
表示在session在cookie标志
通过request中cookie获取sessionid对应的值,此值就是 session_key
middleware AuthenticationMiddleware判定SessionMiddleware中打包的 request.session如果没有触发后续
的操作,正常是存在session,开始生成 request.user
def
get_user
(
request
)
:
if
not
hasattr
(
request
,
'_cached_user'
)
:
request
.
_cached_user
=
auth
.
get_user
(
request
)
return
request
.
_cached_user
class
AuthenticationMiddleware
(
MiddlewareMixin
)
:
def
process_request
(
self
,
request
)
:
assert
hasattr
(
request
,
'session'
)
,
(
"The Django authentication middleware requires session middleware "
"to be installed. Edit your MIDDLEWARE%s setting to insert "
"'django.contrib.sessions.middleware.SessionMiddleware' before "
"'django.contrib.auth.middleware.AuthenticationMiddleware'."
)
%
(
"_CLASSES"
if
settings
.
MIDDLEWARE
is
None
else
""
)
request
.
user
=
SimpleLazyObject
(
lambda
:
get_user
(
request
)
)
真正获取user是
auth.get_user(request)
def
get_user
(
request
)
:
"""
Return the user model instance associated with the given request session.
If no user is retrieved, return an instance of `AnonymousUser`.
"""
from
.
models
import
AnonymousUser
user
=
None
try
:
user_id
=
_get_user_session_key
(
request
)
backend_path
=
request
.
session
[
BACKEND_SESSION_KEY
]
except
KeyError
:
pass
else
:
if
backend_path
in
settings
.
AUTHENTICATION_BACKENDS
:
backend
=
load_backend
(
backend_path
)
user
=
backend
.
get_user
(
user_id
)
# Verify the session
if
hasattr
(
user
,
'get_session_auth_hash'
)
:
session_hash
=
request
.
session
.
get
(
HASH_SESSION_KEY
)
session_hash_verified
=
session_hash
and
constant_time_compare
(
session_hash
,
user
.
get_session_auth_hash
(
)
)
if
not
session_hash_verified
:
request
.
session
.
flush
(
)
user
=
None
return
user
or
AnonymousUser
(
)
def
_get_user_session_key
(
request
)
:
# This value in the session is always serialized to a string, so we need
# to convert it back to Python whenever we access it.
return
get_user_model
(
)
.
_meta
.
pk
.
to_python
(
request
.
session
[
SESSION_KEY
]
)
尝试通过session中附加的信息,获取user_id,而后获取user,完成request中融入 user信息。
Django Https
来由是Django SECURE_PROXY_SSL_HEADER 参数。一般服务都是 cluster,如果要做Https,证书都是配在LB上,基本访问:
Customer --https--> LB --http--> Service
在service访问时有类似的auth验证,扫码登陆,结果service 拿到的 request从LB流入,是http 的协议,
转而扫码登陆后redirect的原始的 链接就是 http的地址。此时如果LB上配置了 http 强制跳转 https即OK,倘若没有,则新用户https访问,跳转扫码登陆页,扫码后转入了 http访问。
其实Django提供了解决方法:
# Django setting 中设置
SECURE_PROXY_SSL_HEADER
=
(
'HTTP_X_FORWARDED_PROTOCOL'
,
'https'
)
原文说明:
A tuple representing a HTTP header/value combination that signifies a request is secure. This controls the behavior of the request object’s is_secure() method.
By default, is_secure() determines if a request is secure by confirming that a requested URL uses https://. This method is important for Django’s CSRF protection, and it may be used by your own code or third-party apps.
If your Django app is behind a proxy, though, the proxy may be “swallowing” the fact that a request is HTTPS, using a non-HTTPS connection between the proxy and Django. In this case, is_secure() would always return False – even for requests that were made via HTTPS by the end user.
In this situation, configure your proxy to set a custom HTTP header that tells Django whether the request came in via HTTPS, and set SECURE_PROXY_SSL_HEADER so that Django knows what header to look for.
Django setting中包含参数特别多,建议查看官网 Django Setting 通篇了解下。