HoneyHoney

Python 不要使用可变类型作为参数的默认值

Posted on Wednesday, September 18, 2019 by 醉·醉·鱼 and labeled under python

《流畅的python》一书中有一小节提到过，不要使用可变类型作为参数的默认值。碰巧自己刷题的时候没有意识到这个，成功踩雷。在python里面，变量都是对象的引用而已。在下面的例子中，修改一个变量，另外一个变量也会变，因为他们其实是同一个对象。


def f(v = []):
    return v

a = f()
b = f()

a.append(1)

print(a == b) # don't use mutable obj as default param
print(a is b)

c = f()
d = f()

c += [1]
print(id(c))
print(id(d))
print(c == d)
print (c is d)

虽然说可以用 a = a + [1] 来避免，但还是不要用可变类型作为参数默认值。

Python, Ruby Thread Safe?

Posted on Friday, June 28, 2019 by 醉·醉·鱼 and labeled under python , ruby

最近在刷python的题，遇到经典的银行提款的问题。做完以后，想在Ruby上面也实验一番，进而发现了更多好玩的知识点，略微整理一下。

问题回顾

从银行账户里面取钱和存钱，多线程操作，看是否会导致账户余额出错。按道理来说，最后应该还是1000块钱。



class BankAccount(object):

    def __init__(self):
        self.balance = 0
        pass

    def withdraw(self, amount):           
        self.balance -= amount

    def deposit(self, amount):
        self.balance += amount


    def test_can_handle_concurrent_transactions(self):
        account = BankAccount()
        account.open()
        account.deposit(1000)

    self.adjust_balance_concurrently(account)

    self.assertEqual(account.get_balance(), 1000)

    def adjust_balance_concurrently(self, account):
        def transact():
            account.deposit(5)
            time.sleep(0.001)
            account.withdraw(5)

        # Greatly improve the chance of an operation being interrupted
        # by thread switch, thus testing synchronization effectively
        try:
            sys.setswitchinterval(1e-12)
        except AttributeError:
        # For Python 2 compatibility
            sys.setcheckinterval(1)

        threads = [threading.Thread(target=transact) for _ in range(1000)]
        for thread in threads:
            thread.start()
        for thread in threads:
            thread.join()

结果，上面这个python测试代码通常是失败的。根据https://opensource.com/article/17/4/grok-gil，`+=`不是原子性操作，中间可以GIL切换到其他线程上面去操作了，进而导致问题。要想避免这个问题，引入lock即以。



lock = threading.Lock()

class BankAccount(object):

    def __init__(self):
        self.balance = 0
        pass

    def withdraw(self, amount):           
        with lock:
            self.balance -= amount

    def deposit(self, amount):
        with lock:
            self.balance += amount

但是，Ruby代码却不会有这样的问题。根据https://www.jstorimer.com/blogs/workingwithcode/8100871-nobody-understands-the-gil-part-2-implementation，all methods in MRI are atomic. 所有操作都是原子性，没法在分割的，即不会出现中间GIL 切换到其他现成上面去。




class BankAccount
  attr_accessor :balance

  def initialize
    @balance = 0
    @semapore = Mutex.new
  end
  
  def deposit(amount)
    @balance += amount
  end

  def withdraw(amount)
    @balance -= amount
  end
  
end


account = BankAccount.new
account.deposit(1000)

threads = []

1000.times do
  threads << Thread.new do
    account.withdraw(1)
  end
end

threads.each(&:join)

p account.balance

那，有没有办法在Ruby里面去模拟这种情况呢？有，而且应用还颇为广泛。
如下面这段代码，先把balance缓存起来，然后sleep，再做操作，就会出现GIL切换线程的现象，进而出现线程安全问题。这种情况在很多地方都会出现，中间的sleep常常表现为系统进行其他的操作，但时间不可预知。回过头再把缓存的数据进行操作的时候，其他线程早就已经更改了。这种情况最典型的例子就是秒杀时候inventory的问题。

将注释去掉，引入`semapore.synchronize`即可。


  def withdraw(amount)
    # @semapore.synchronize do
      balance = @balance
      sleep rand(10) / 10000.0
      @balance = balance - amount
    # end
  end

IE doesn't redirect 302 from POST > redirect > GET?

Posted on Friday, November 09, 2018 by 醉·醉·鱼 and labeled under IE

You may expect IE will redirect 302 for POST to a GET request. The truth is IE does it. However, the default developer tools sucks.

As you could see from the screenshot below, you will always see POST after 302 response in IE developer tools.

If you see details for these request, you will see both POST and GET requests. How could it be?

While, if you use other network monitor tools to monitor the network, you will see only GET request.

It will bring developers into wrong directions as they would think it's the HTTP method error. For example: https://stackoverflow.com/questions/28513449/ie-ignores-303-redirect-in-post-redirect-get-scenario

And I would share another IE bug. If there is underscore in your domain, no cookies will be set.

See https://blogs.msdn.microsoft.com/ieinternals/2009/08/20/internet-explorer-cookie-internals-faq/

Ruby: How to Override Class Method With a Module

Posted on Tuesday, September 04, 2018 by 醉·醉·鱼 and labeled under ruby

You have a class with a class method. Write a module that, when included, will override that class method.

在这篇文章里面已经介绍了一种方法 http://tech.tulentsev.com/2012/02/ruby-how-to-override-class-method-with-a-module/，但我还可以分享另外的方法。

方法一：用instance_eval。原文中的方法。


module BooModule
  def self.included base
    base.instance_eval do
      def bar
        puts "module"
      end
    end
  end
end

class KlassC

  def self.bar
    puts 'class'
  end
end

KlassC.send(:include, BooModule)

KlassC.bar #=> module
p KlassC.ancestors

方法二：用 class << base ; end


module TooModule
  def self.included base
    class << base
      def bar
        puts "module"
      end
    end
  end
end

class KlassD

  def self.bar
    puts 'class'
  end
end

KlassD.send(:include, TooModule)

KlassD.bar #=> module
p KlassD.ancestors

方法三：用prepend。如果你知道extend实际上就是class << base; include ClassMethods end的话，你就很好理解为啥prepend在这里可以起作用了。前两者都是直接修改类方法，而这直接通过修改继承链的方法去覆盖类方法。


module GooModule
  def self.included base
    class << base
      prepend ClassMethods
    end
  end

  module ClassMethods
    def bar
      puts "module"
    end
  end
end

class KlassE
  def self.bar
    puts 'class'
  end
end

KlassE.send(:include, GooModule)

KlassE.bar #=> module
p KlassE.ancestors

能够直接用prepend么？不行。你可以试试下面的例子。虽然说，被prepend的module会放到class的继承链的下方，但实际上，类方法却不受影响。


module FooModule
  def self.included base
    base.extend ClassMethods
  end

  def self.prepended base
    base.extend ClassMethods
  end

  module ClassMethods
    def bar
      puts "module"
    end
  end
end

class KlassA
  include FooModule

  def self.bar
    puts 'class'
  end
end


KlassA.bar #=> class
p KlassA.ancestors

class KlassB
  prepend FooModule

  def self.bar
    puts 'class'
  end
end


KlassB.bar #=> class
p KlassB.ancestors

Show test case name in verbose mode for Minitest 5.11.3

Posted on Tuesday, September 04, 2018 by 醉·醉·鱼 and labeled under ruby

最新的5.11.3 minitest不知道为什么，在verbose模式下，把对应的测试用例的名字给抹掉了。这样的话，导致分析测试用例performance的时候，非常得不友好。

https://github.com/seattlerb/minitest/blob/master/lib/minitest.rb#L617


    def record result # :nodoc:
      io.print "%.2f s = " % [result.time] if options[:verbose]
      io.print result.result_code
      io.puts if options[:verbose]
    end
  end

将上述代码替换成下面代码，就可以看到某个test用掉多少时间。


    def record result # :nodoc:
      io.print "%s#%s = %.2f s = " % [result.klass, result.name, result.time] if options[:verbose]
      io.print result.result_code
      io.puts if options[:verbose]
    end

输出

Run options: -n test_viewing_transaction --verbose --seed 39465

# Running:

TransactionsControllerTest#test_viewing_transaction = 6.13 s = .

Finished in 7.851562s, 0.1274 runs/s, 0.1274 assertions/s.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

List docker contrainer IP

Posted on Thursday, August 30, 2018 by 醉·醉·鱼 and labeled under docker

罗列所有container的IP地址


docker ps -q | xargs -n 1 docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} {{ .Name }}'

单独monitor某个contrainer的logs


docker ps -a | grep payments | sed 's/ .*//' | xargs docker logs -f

Implement DatabaseCleaner in Rails 4

Posted on Tuesday, August 21, 2018 by 醉·醉·鱼 and labeled under Mysql , Rails , ruby

DatabaseCleaner上面的example过时了。按照示例去做的话，会报错 unknown method 'before'。在github上找到了答案，还是比较简单的。


class ActiveSupport::TestCase
  include FactoryGirl::Syntax::Methods

  ActiveRecord::Migration.check_pending!
  DatabaseCleaner.strategy = :truncation
  DatabaseCleaner.logger = Rails.logger
  setup { DatabaseCleaner.start }
  teardown { DatabaseCleaner.clean }
end

配置完成以后，就可以在MYSQL去monitor query，看具体是如何操作数据库的。

mysql> SHOW VARIABLES LIKE "general_log%";

+------------------+----------------------------+
| Variable_name    | Value                      |
+------------------+----------------------------+
| general_log      | OFF                        |
| general_log_file | /var/run/mysqld/mysqld.log |
+------------------+----------------------------+

mysql> SET GLOBAL general_log = 'ON';

观察log


tail -f -n300 /var/run/mysqld/mysqld.log

最后，重置改动。


mysql> SET GLOBAL general_log = 'OFF';

https://stackoverflow.com/questions/568564/how-can-i-view-live-mysql-queries
https://github.com/metaskills/minitest-spec-rails/issues/44#issuecomment-244155657

HoneyHoney

醉醉鱼·涂鸦志

Python 不要使用可变类型作为参数的默认值

Python, Ruby Thread Safe?

问题回顾

IE doesn't redirect 302 from POST > redirect > GET?

Ruby: How to Override Class Method With a Module

You have a class with a class method. Write a module that, when included, will override that class method.

Show test case name in verbose mode for Minitest 5.11.3

List docker contrainer IP

Implement DatabaseCleaner in Rails 4

About Me

Search

Blog Archive

Label Cloud