169. 多数元素

题目

206. 反转链表

题目

121. 买卖股票的最佳时机

题目

We used to keep private credentials on production servers without any protection or encryption. Well, luckily we don’t have any leak but this practice is not recommended for both security and easy of use reasons.

Since AWS finally provides KMS(Key Management Service) in our local region, we try to encrypt every private credentials by KMS and store them on S3.

TBD

Partitioning refers to splitting what is logically one large table inot smaller physical pieces.

Currently, PostgreSQL supports partitioning via table inheritance. Each partition must be created as a child table of a single parent table. The parent table itself is normally empty; It exists just to represent the entire data set.

There are two forms of partitioning can be implemented in PostgreSQL:

• Range Partitioning

  ​ The table is partitioning into “range” defined by a key column or a set of columns, with no overlap between the ranges of values assigned to different partitions. eg. partition by date ranges or by identifiers.

• List Partitioning

  ​ The table is partitioned by explicitly listing which key values appear in each partition.

Implementing Partitioning

1. Create the “master” / “parent” table, from which all the partitions will inherit.

   ​ This table will not contain any data. Do not define any check on this table, unless you intend them to be applied equally to all partitions. There is no point in defining any indexes or unique constraints on it either.

2. Create “child” tables that each inherit form the master table. Normally, these tables will not add any columns to the set inherited from the master.

3. Add table constraints to the partition tables to define the allowed key values in each partitions.

   ​ Ensure that the constraints guarantee that there is no overlap between the key values premitted in different partitions. And there is no difference in syntax between range and list partitioning.

4. Create indexes on column(s) for each partitions.

5. Optionally, define a trigger or rule to redirect data inserted into the master table to the appropriate partition.

6. Ensure hte constraint_exclusion configuration parameter is not disabled in postgresql.conf. If it is, queries will not be optimized as desired.

Trigger

As we are creating new table and hopping data insered to right partition, a trigger function and a trigger are needed.

I’ll come across many hg branch-switching or log searching tasks during my work. Using bash functions and awk greatly reduce the time I spend on dealing with these tasks. So let’s see what these functions can do to help me improve my productivity.

bash functions

Bash functions are scripts like alias, but can do much a lot than aliasThe first kind of usages of function is to run several scripts continuously, the same as && I guess:

1
2
3
4

function run {
    source ~/Develop/django/bin/activate
    ./manage.py runserver
}

I activate django virtural enviroment first and then run django serve.

Functions, like in any other languages, can take parameters and returns a result. So I can use this ability to do a liitle more complex work:

1
2

function ba { hgba | grep "$1" }
# hgba is an alias for hg branches

I can just type ba release then get current release branch info.

Append:

I made some updates to the ba function and make it auto copying the branch result to my clipboard:

1
2

function ba { var="$(hgba | grep $1)" && echo $var | awk -F ':' END{print} | awk -F ':' '{print $NF}' | tr -d '\n' | pbcopy && echo $var }

awk

awk is a command I just know recently. I need to process a bunch of logs and analyze them. What I used to do is download the logs, grep things I want into a txt file and then process the txt file with python.

Flow Through The Request/Response Cycle

Tastypie can be thought of as a set of class-based view that provide the API functionality. All routing/middleware/response-handling aspectss are the same as a typical Django app. Where the differs is in the view itself.

Walking through what a GET request to a list endpoint looks like:

• The Resource.urls are checked by Django’s url resolvers.

• On a match for the list view, Resource.wrap_view('dispatch_list') is called. wrap_view provides basic error handling & allows for returning serilized errors.

• Because dispatch_list was passed to wrap_view, Resource.dispatch_list is called next. This is a thin wrapper around Resource.dispatch.

• dispatch does a bunch of havy lifting. It ensures:

  • the requested HTTP method is in allowed_methos (method_check).
  • the class has a method that can handle the request(get_list)
  • the user is authenticated(is_authenticated)
  • the user has no exceeded their throttle(throttle_check).

  At this point, dispatch actually calls the requested method (get_list).

• get_list does the actual work of API. It does:

  • A fetch of the available objects via Resource.obj_get_list. In the case of ModelResource, this builds the ORM filters to apply (ModelResource.build_filters). It then gets the QuerySet via ModelResource.get_object_list (which performs Resource.authorized_read_list to possibly limit the set the user can work with) and applies the built filters to it.
  • It then sorts the objects based on user input (ModelResource.apply_sorting).
  • Then it paginates the results using the supplied Paginator & pulls out the data to be serialized.
  • The objects in the page have full_dehydrate applied to each of them, causing Tastypie to traslate the raw object data into the fields the endpoint supports.
  • Finally, it calls Resource.create_response.

• create_response is a shortcut method that:

  • Determines the desired response format (Resource.determine_format).
  • Serializes the data given to it in the proper format.
  • Returns a Django HttpResponse (200 OK) with the serialized data.

• We bubble back up the call stack to dispatch. The last thing dispatch does is potentially store that a request occured for future throttling (Resource.log_throttled_access) then either returns the HttpResponse or wraps whatever data came back in a response (so Django doesn’t freak out).

Resources in Tastypie

Resources are the heart of Tastypie. By defining a resource we can actually convert a model into an API stream. The data is automatically converted into API response.

Understanding the process of creating a resource.

1. Import ModelResource from Tastypie.
2. Import models from services app
3. Create custom resource by inheriting ModelResource and link app model in inner Meta class of resource.

Add API URL in the urls.py of app.

Dehydrating the JSON data

Dehydration in Tastypie means making alterations before sending data to the client. Suppose we need to send capitalized product names instead of small letters. Now we see two kinds of dehydrate methods.

Dehydrate_field method

This dehydrate_field is uesd to modify field on the response JSON.

Dehydrate method

Dehydrate method is useful for aadding additional fields to bundle (response data).

Similarly using hydrate method we can alter the bundle data which is generated from request at the time of PUT or POST methods.

Django Manager

Django 里会为每一个 model 生成一个 Manager，默认名字为 objects，一般情况下对 model 进行的处理都是通过 model.objects.XXX( ) 来进行的。其实是调用了 model 的 manager 的方法，而 manager 之中的方法是 QuerySet 方法的代理，QuerySet 方法是对数据库操作的封装。

eg.

1
2
3
4
5

from django.db import models

class Person(models.Model):
	...
	people = models.Manager()

上面这个 model，Person.objects会产生一个AttributeError，但是Person.people就可以正常操作。因为默认的 manager 已经变成 people，objects 这个 manager 没有重新声明，不起作用。

自定义 Manager

通常需要自定义 manager 的情况有两点:

1. 需要修改/扩展 Django 的 manager 方法
2. 需要修改返回的 QuerySet

默认 Manager

如果使用自定义的 manager 需要注意的是，Django 将 model 中定义的第一个 manager 认为是默认 manager，而且 Django 框架中会用到默认 manager。

笨方法是使用自定义 manager 的时候，对于 model 依然提供 objects 这个默认 manager，并放在第一个。

eg.

1
2
3
4
5
6

class Book(models.Model):
	title = models.CharField(max_length=100)
	author = models.CharField(max_length=50)
	
	objects = models.Manager() # default manager
	custom_objects = CustomBOokManager() # custom manager

source

To handle web forms we use Flask-WTF . So we need to write a config file (file config.py):

1
2

WTF_CSRF_ENABLED = True
SECRET_KEY = 'you-will-never-guess'

And then you need to use this config (file app/__init__.py):

1
2
3
4
5
6

from flask import Flask

app = Flask(__name__)
app.config.from_object('config')

from app import views

Let’s build a simple form (file app/forms.app):

1
2
3
4
5
6
7

from flask.ext.wtf import Form
from wtforms import StringField, BooleanField
from wtforms.validators import DataRequired

class LoginForm(Form):
    openid = StringField('openid', validators=[DataRequired()])
    remember_me = BooleanField('remember_me', default=False)

The DataRequired() is a validator that checks the field is empty or not.

After that, we need a HTML page to show the form (file app/templates/login.html):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<!-- extend from base layout -->
{% extends "base.html" %}

{% block content %}
  <h1>Sign In</h1>
  <form action="" method="post" name="login">
      {{ form.hidden_tag() }}
      <p>
          Please enter your OpenID:<br>
          {{ form.openid(size=80) }}<br>
      </p>
      <p>{{ form.remember_me }} Remember Me</p>
      <p><input type="submit" value="Sign In"></p>
  </form>
{% endblock %}

The final step is to code a view function that renders the template and receiving data from form (file app/views.py):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

from flask import render_template, flash, redirect
from app import app
from .forms import LoginForm

# index view function suppressed for brevity

app.route('/login', methods=['GET', 'POST'])
def login():
    form = LoginForm()
    if form.validate_on_submit():
        flash('Login requested for OpenID="%s", remember_me=%s' %
              (form.openid.data, str(form.remember_me.data)))
        return redirect('/index')
    return render_template('login.html', 
                           title='Sign In',
                           form=form)