Python -- pycurl, mysqlclient, Django

系统 1542 0

前言

本不想多说什么,年后在公司亲历一段别样经验,不想吐槽,只因吐槽不能改变任何现状。小姐心态,寡妇待遇,妇联追求,一份技术工作做出弯腰,低头,下跪,也是醉了。今年大环境不好,大厂裁员,人才过盛,好在自己还是去了符合自己意愿的公司。
新工作快一个月,主要工作技术内容是一个Django的小东西,首先是需要从Python2 迁移至Python3,简单带点重构。

Python2 迁移Python3 运行环境

服务run在docker里,启动方式里面没有嵌入太多环境变量,基本基础环境。但是Django自身依赖一些库环境需要解决。

pycurl

安装 pycurl可能会遇到 ssl lib的Error,建议执行:

            
              
                export
              
               PYCURL_SSL_LIBRARY
              
                =
              
              openssl

              
                export
              
               LDFLAGS
              
                =
              
              -L/usr/local/opt/openssl/lib

              
                export
              
               CPPFLAGS
              
                =
              
              -I/usr/local/opt/openssl/include


              
                # 执行上述环境变量设之后,再安装
              
              
pip 
              
                install
              
               pycurl

            
          

mysqlclient

mysqlclient比较坑,安装时候会抛出: library not found for -lssl

            
              running build_ext
building 'MySQLdb._mysql' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/MySQLdb
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -Dversion_info=(1,4,2,'post',1) -D__version__=1.4.2.post1 -I/usr/local/opt/mysql@5.7/include/mysql -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c MySQLdb/_mysql.c -o build/temp.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.o
gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.o -L/usr/local/opt/mysql@5.7/lib -lmysqlclient -lssl -lcrypto -o build/lib.macosx-10.9-x86_64-3.7/MySQLdb/_mysql.cpython-37m-darwin.so
ld: library not found for -lssl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1

            
          

mysqlclient 官网上给出了比较清晰的安装方式https://github.com/PyMySQL/mysqlclient-python

debian

centos和debian下还比较好解决,直接系统安装 dev包即可

            
              
                # 先安装
              
              
                apt-get
              
              
                install
              
               python-dev default-libmysqlclient-dev

              
                # 再安装mysqlclient
              
              
pip 
              
                install
              
               mysqlclient

            
          

MacOS

MacOS下问题比较多,官网上给的很清楚,mysql-connector-c有bug

            
              
                # 首先安装mysql
              
              
brew install mysql@
              
                5.7
              
              
                # 安装 mysql-connector-c
              
              
brew install mysql
              
                -
              
              connector
              
                -
              
              c

            
          

编辑/usr/local/bin/mysql_config 确保:
libs="-L$pkglibdir"

libs="$libs -lmysqlclient -lssl -lcrypto"

            
              
                # 修改/usr/local/bin/mysql_config
              
              
vim /usr/local/bin/mysql_config

              
                # 设置openssl环境变量
              
              
                export
              
               PATH
              
                =
              
              
                "/usr/local/opt/openssl/bin:
                
                  $PATH
                
                "
              
              
                export
              
               LDFLAGS
              
                =
              
              
                "-L/usr/local/opt/openssl/lib"
              
              
                export
              
               CPPFLAGS
              
                =
              
              
                "-I/usr/local/opt/openssl/include"
              
              
                # 再安装mysqlclient
              
              
pip 
              
                install
              
               mysqlclient

            
          

基本环境设置好,直接安装依赖即可

Django

貌似最初开始接触Python就是Django,从Python2至Python3也不是一个容易的过程

Django 版本差异

1,python2.x下,Django最多支持版本应该是1.11,目前Django 都2.2了,中间相差太多
Python3下建议使用高版本
2,djangorestframework是Django用来做纯RESTFull API的,官网介绍如下:
https://www.django-rest-framework.org/#quickstart
并且新版本的设计使用方式,偏向于router register viewset形式,配合action:
https://www.django-rest-framework.org/api-guide/viewsets/
注意
1,低版本djangorestframework不支持 action写法,建议使用最新版本
2,router 写法需要严格村从RESTFull形式,router.register prefix中不支持 'pre/fix'形式,action中url_path也不支持 '/'和'.'

APPEND_SLASH

Django 在路由传统路由匹配时,会有结尾 ‘/’的区分,router.register和action中都是默认以‘/’结尾,

            
              urlpatterns 
              
                =
              
              
                [
              
              
    url
              
                (
              
              r
              
                '^user/$'
              
              
                ,
              
               views
              
                .
              
              NodeView
              
                .
              
              as_view
              
                (
              
              
                )
              
              
                )
              
              
                ,
              
              
                # url(r'^user$', views.NodeView.as_view()),
              
              
                ]
              
            
          

以上两种实质是不一样的uri,但是Django默认APPEND_SLASH=True的,当:

            
              MIDDLEWARE 
              
                =
              
              
                [
              
              
                .
              
              
                .
              
              
                .
              
              
                'django.middleware.common.CommonMiddleware'
              
              
                ,
              
              
                .
              
              
                .
              
              
                .
              
              
                ]
              
            
          

middleware 中开启CommonMiddleware,会对用户访问 http://doamin/user 进行自动redirect 至 http://doamin/user/
迁移过程中,很多url不规范,都是 r'^user$' 形式,整合到router.register形式就会有重定向问题,不仅如此,在主urls.py文件中:

            
              urlpatterns 
              
                =
              
              
                [
              
              
                .
              
              
                .
              
              
                .
              
              
    url
              
                (
              
              r
              
                '.*'
              
              
                ,
              
               views
              
                .
              
              main_view
              
                )
              
              
                ,
              
              
                # 没收所有没有匹配上的
              
              
                ]
              
              

@login_required

              
                def
              
              
                main_view
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                return
              
               TemplateResponse
              
                (
              
              request
              
                ,
              
              
                'index.html'
              
              
                )
              
            
          

这种形式构造,原本 http://doamin/user 完全可以urlpatterns中未匹配任何,之后缺少 结尾’/'后重定向匹配http://doamin/user/,
现在被 r'.*' 没收,返回index界面,而且还需要是login的,否则跳转到login界面

proxy

django-proxy库用来对一个request进行proxy转发,在多数整合系统中用来做一个简单gateway很是便捷,Django自带的Redirect是
本域内的重定向,并非转发。https://github.com/mjumbewu/django-proxy,用法也很清晰。
关闭APPEND_SLASH=False,对上面自动重定向问题进行手动处理,自动匹配所有user prefix下的所有未具有’/'请求:

            
              
                from
              
               proxy
              
                .
              
              views 
              
                import
              
               proxy_view


              
                def
              
              
                main_view
              
              
                (
              
              request
              
                )
              
              
                :
              
              
    path
              
                ,
              
               host 
              
                =
              
               request
              
                .
              
              path
              
                ,
              
               request
              
                .
              
              get_host
              
                (
              
              
                )
              
              
                if
              
               path
              
                .
              
              startswith
              
                (
              
              
                '/user/'
              
              
                )
              
              
                and
              
              
                not
              
               path
              
                .
              
              endswith
              
                (
              
              
                '/'
              
              
                )
              
              
                :
              
              
                return
              
               proxy_view
              
                (
              
              request
              
                ,
              
               f
              
                'http://{host}{path}/'
              
              
                )
              
              

@login_required

              
                def
              
              
                logined_view
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                return
              
               TemplateResponse
              
                (
              
              request
              
                ,
              
              
                'index.html'
              
              
                )
              
            
          

原有 r'.*' 在urlpatterns最末尾,访问http://doamin/user/直接就匹配上,访问http://doamin/user被 r'.*' 匹配,而后对
request中的path进行判定,如果是/user/起始前缀的,并且没有’/‘结尾,补充’/'后重新访问。
注意
此处用proxy,并非redirect,因为关于APPEND_SLASH的说明中给出,对于APPEND_SLASH会丢失 POST访问,所以此处用proxy一并解决

Django User

默认Django在 request 中获取user,即为获取User model中对应的访问user

            
              
                def
              
              
                response_func
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                print
              
              
                (
              
              request
              
                .
              
              user
              
                )
              
            
          

背后跟session关联的,关于Django session的设定,Django session 官网给了比较详细的介绍

            
              MIDDLEWARE 
              
                =
              
              
                [
              
              
                .
              
              
                .
              
              
                .
              
              
                'django.contrib.sessions.middleware.SessionMiddleware'
              
              
                ,
              
              
                .
              
              
                .
              
              
                .
              
              
                'django.contrib.auth.middleware.AuthenticationMiddleware'
              
              
                ,
              
              
                .
              
              
                .
              
              
                .
              
              
                ]
              
            
          

中通过利用session确定用户。

            
              
                class
              
              
                SessionMiddleware
              
              
                (
              
              MiddlewareMixin
              
                )
              
              
                :
              
              
                def
              
              
                __init__
              
              
                (
              
              self
              
                ,
              
               get_response
              
                =
              
              
                None
              
              
                )
              
              
                :
              
              
        self
              
                .
              
              get_response 
              
                =
              
               get_response
        engine 
              
                =
              
               import_module
              
                (
              
              settings
              
                .
              
              SESSION_ENGINE
              
                )
              
              
        self
              
                .
              
              SessionStore 
              
                =
              
               engine
              
                .
              
              SessionStore

    
              
                def
              
              
                process_request
              
              
                (
              
              self
              
                ,
              
               request
              
                )
              
              
                :
              
              
        session_key 
              
                =
              
               request
              
                .
              
              COOKIES
              
                .
              
              get
              
                (
              
              settings
              
                .
              
              SESSION_COOKIE_NAME
              
                )
              
              
        request
              
                .
              
              session 
              
                =
              
               self
              
                .
              
              SessionStore
              
                (
              
              session_key
              
                )
              
            
          

settings.SESSION_ENGINE 设置了session的存储引擎,默认 django.contrib.sessions.backends.db 就是用db
settings.SESSION_COOKIE_NAME默认的名称 sessionid 表示在session在cookie标志
通过request中cookie获取sessionid对应的值,此值就是 session_key
middleware AuthenticationMiddleware判定SessionMiddleware中打包的 request.session如果没有触发后续
的操作,正常是存在session,开始生成 request.user

            
              
                def
              
              
                get_user
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                if
              
              
                not
              
              
                hasattr
              
              
                (
              
              request
              
                ,
              
              
                '_cached_user'
              
              
                )
              
              
                :
              
              
        request
              
                .
              
              _cached_user 
              
                =
              
               auth
              
                .
              
              get_user
              
                (
              
              request
              
                )
              
              
                return
              
               request
              
                .
              
              _cached_user



              
                class
              
              
                AuthenticationMiddleware
              
              
                (
              
              MiddlewareMixin
              
                )
              
              
                :
              
              
                def
              
              
                process_request
              
              
                (
              
              self
              
                ,
              
               request
              
                )
              
              
                :
              
              
                assert
              
              
                hasattr
              
              
                (
              
              request
              
                ,
              
              
                'session'
              
              
                )
              
              
                ,
              
              
                (
              
              
                "The Django authentication middleware requires session middleware "
              
              
                "to be installed. Edit your MIDDLEWARE%s setting to insert "
              
              
                "'django.contrib.sessions.middleware.SessionMiddleware' before "
              
              
                "'django.contrib.auth.middleware.AuthenticationMiddleware'."
              
              
                )
              
              
                %
              
              
                (
              
              
                "_CLASSES"
              
              
                if
              
               settings
              
                .
              
              MIDDLEWARE 
              
                is
              
              
                None
              
              
                else
              
              
                ""
              
              
                )
              
              
        request
              
                .
              
              user 
              
                =
              
               SimpleLazyObject
              
                (
              
              
                lambda
              
              
                :
              
               get_user
              
                (
              
              request
              
                )
              
              
                )
              
            
          

真正获取user是 auth.get_user(request)

            
              
                def
              
              
                get_user
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                """
    Return the user model instance associated with the given request session.
    If no user is retrieved, return an instance of `AnonymousUser`.
    """
              
              
                from
              
              
                .
              
              models 
              
                import
              
               AnonymousUser
    user 
              
                =
              
              
                None
              
              
                try
              
              
                :
              
              
        user_id 
              
                =
              
               _get_user_session_key
              
                (
              
              request
              
                )
              
              
        backend_path 
              
                =
              
               request
              
                .
              
              session
              
                [
              
              BACKEND_SESSION_KEY
              
                ]
              
              
                except
              
               KeyError
              
                :
              
              
                pass
              
              
                else
              
              
                :
              
              
                if
              
               backend_path 
              
                in
              
               settings
              
                .
              
              AUTHENTICATION_BACKENDS
              
                :
              
              
            backend 
              
                =
              
               load_backend
              
                (
              
              backend_path
              
                )
              
              
            user 
              
                =
              
               backend
              
                .
              
              get_user
              
                (
              
              user_id
              
                )
              
              
                # Verify the session
              
              
                if
              
              
                hasattr
              
              
                (
              
              user
              
                ,
              
              
                'get_session_auth_hash'
              
              
                )
              
              
                :
              
              
                session_hash 
              
                =
              
               request
              
                .
              
              session
              
                .
              
              get
              
                (
              
              HASH_SESSION_KEY
              
                )
              
              
                session_hash_verified 
              
                =
              
               session_hash 
              
                and
              
               constant_time_compare
              
                (
              
              
                    session_hash
              
                ,
              
              
                    user
              
                .
              
              get_session_auth_hash
              
                (
              
              
                )
              
              
                )
              
              
                if
              
              
                not
              
               session_hash_verified
              
                :
              
              
                    request
              
                .
              
              session
              
                .
              
              flush
              
                (
              
              
                )
              
              
                    user 
              
                =
              
              
                None
              
              
                return
              
               user 
              
                or
              
               AnonymousUser
              
                (
              
              
                )
              
              
                def
              
              
                _get_user_session_key
              
              
                (
              
              request
              
                )
              
              
                :
              
              
                # This value in the session is always serialized to a string, so we need
              
              
                # to convert it back to Python whenever we access it.
              
              
                return
              
               get_user_model
              
                (
              
              
                )
              
              
                .
              
              _meta
              
                .
              
              pk
              
                .
              
              to_python
              
                (
              
              request
              
                .
              
              session
              
                [
              
              SESSION_KEY
              
                ]
              
              
                )
              
            
          

尝试通过session中附加的信息,获取user_id,而后获取user,完成request中融入 user信息。

Django Https

来由是Django SECURE_PROXY_SSL_HEADER 参数。一般服务都是 cluster,如果要做Https,证书都是配在LB上,基本访问:

            
              Customer  --https-->   LB  --http-->  Service

            
          

在service访问时有类似的auth验证,扫码登陆,结果service 拿到的 request从LB流入,是http 的协议,
转而扫码登陆后redirect的原始的 链接就是 http的地址。此时如果LB上配置了 http 强制跳转 https即OK,倘若没有,则新用户https访问,跳转扫码登陆页,扫码后转入了 http访问。

其实Django提供了解决方法:

            
              
                # Django setting 中设置
              
              
SECURE_PROXY_SSL_HEADER 
              
                =
              
              
                (
              
              
                'HTTP_X_FORWARDED_PROTOCOL'
              
              
                ,
              
              
                'https'
              
              
                )
              
            
          

原文说明:
A tuple representing a HTTP header/value combination that signifies a request is secure. This controls the behavior of the request object’s is_secure() method.

By default, is_secure() determines if a request is secure by confirming that a requested URL uses https://. This method is important for Django’s CSRF protection, and it may be used by your own code or third-party apps.

If your Django app is behind a proxy, though, the proxy may be “swallowing” the fact that a request is HTTPS, using a non-HTTPS connection between the proxy and Django. In this case, is_secure() would always return False – even for requests that were made via HTTPS by the end user.

In this situation, configure your proxy to set a custom HTTP header that tells Django whether the request came in via HTTPS, and set SECURE_PROXY_SSL_HEADER so that Django knows what header to look for.

Django setting中包含参数特别多,建议查看官网 Django Setting 通篇了解下。


更多文章、技术交流、商务合作、联系博主

微信扫码或搜索:z360901061

微信扫一扫加我为好友

QQ号联系: 360901061

您的支持是博主写作最大的动力,如果您喜欢我的文章,感觉我的文章对您有帮助,请用微信扫描下面二维码支持博主2元、5元、10元、20元等您想捐的金额吧,狠狠点击下面给点支持吧,站长非常感激您!手机微信长按不能支付解决办法:请将微信支付二维码保存到相册,切换到微信,然后点击微信右上角扫一扫功能,选择支付二维码完成支付。

【本文对您有帮助就好】

您的支持是博主写作最大的动力,如果您喜欢我的文章,感觉我的文章对您有帮助,请用微信扫描上面二维码支持博主2元、5元、10元、自定义金额等您想捐的金额吧,站长会非常 感谢您的哦!!!

发表我的评论
最新评论 总共0条评论