Latest Updates: Python RSS

• 

  Pubsubhubbub on Google appspot

  Stii 7:36 am on March 11, 2010 | Comments: 2 Permalink | Reply
  Tags: Afrigator (23), google (8), PHP (7), pubsubhubbub (3), pushpress (2), Python, Wordpress (14)

  It is the weirdest thing. I was having issues subscribing to blogs that is using pubsubhubbub.appspot.com using a Python script. It returned a 500 Internal Server error without fail. So I rewrote the process in PHP and like magic, it is gone… I used PHP and CURL to subscribe to the appspot service. In Python I used the urllib2 library. Somewhere in the back of my mind I vaguely remember appspot not liking urllib2, but I haven’t really checked due to time constraints. I’ll have a look soon, but if you’re going to use Python to subscribe to feeds, I would suggest trying to do so with CURL.

  The good news is that all is good on Afrigator now. All blogs that are using some form of Pubsubhubbub service will be realtime in no time.

  If you’re running on WordPress (not WordPress.com) and you have not yet installed PuSHPress, please do so soon!

  Welcome back! You should subscribe to my RSS feed here.
  You should follow me on Twitter here
  You should follow me on Gatorpeeps here.
  
   

  Reply Click here to cancel reply.

  Name (required)

  Email (required)

  Web Site

   
• 

  The Google Go programming language

  Stii 2:57 pm on November 11, 2009 | Comments: 2 Permalink | Reply
  Tags: c, google (8), google go, java (4), pascal, programming (6), programming languages, Python

  Lately Google announced a wide array of new products and features. The Google Chrome browser, Google Wave, Android and Google Chrome OS are the ones immediately coming to mind. This morning I saw they released Go. Their own experimental programming language.

  Go is not a scripting (a.k.a. interpreted) language, but a compiled language like C or C++. It looks very, very simple compared to C/C++ and according to them it was born from their frustrations with said languages. This has potential to become popular should they drive it sufficiently.

  The syntax of Go looks like a mix of Python, C, Java and Pascal. Have a look at this:

  package main
  
  import (
      "os";
      "flag";  // command line option parser
  )
  
  var omitNewline = flag.Bool("n", false, "don't print final newline")
  
  const (
      Space = " ";
      Newline = "\n";
  )
  
  func main() {
      flag.Parse();   // Scans the arg list and sets up flags
      var s string = "";
      for i := 0; i < flag.NArg(); i++ {
          if i > 0 {
              s += Space
          }
          s += flag.Arg(i)
      }
      if !*omitNewline {
          s += Newline
      }
      os.Stdout.WriteString(s);
  }
  

  I can say this: It looks friendly! I like the sugar.

  Some of the features makes a lot of sense and I hope this will evolve successfully. Just look at the names behind this little experimental project and you’re bound to get excited! I’d love to know what C and C++ stalwarts think.

  PS: Love the origin of the name:

  “Ogle” would be a good name for a Go debugger.

  I concur.
  
   
• 

  Django tests fail on a new project

  Stii 12:16 am on October 16, 2009 | Comments: 4 Permalink | Reply
  Tags: django (4), Python, tdd, unittest

  This might be slightly confusing at first, but actually makes perfect sense if you think about it. If you have a brand new Django project and you run the tests it fails with a number of errors.

  # python manage.py test
  Creating test database...
  Creating table auth_permission
  Creating table auth_group
  Creating table auth_user
  Creating table auth_message
  Creating table django_content_type
  Creating table django_session
  Creating table django_site
  Installing index for auth.Permission model
  Installing index for auth.Message model
  EE..E...EEEEEEE..................
  ======================================================================
  ERROR: test_password_change_fails_with_invalid_old_password (django.contrib.auth.tests.views.ChangePasswordTest)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/Library/Python/2.6/site-packages/django/contrib/auth/tests/views.py", line 156, in test_password_change_fails_with_invalid_old_password
      'new_password2': 'password1',
    File "/Library/Python/2.6/site-packages/django/test/client.py", line 313, in post
      response = self.request(**r)
    File "/Library/Python/2.6/site-packages/django/core/handlers/base.py", line 92, in get_response
      response = callback(request, *callback_args, **callback_kwargs)
    File "/Library/Python/2.6/site-packages/django/contrib/auth/decorators.py", line 78, in __call__
      return self.view_func(request, *args, **kwargs)
    File "/Library/Python/2.6/site-packages/django/contrib/auth/views.py", line 160, in password_change
      }, context_instance=RequestContext(request))
    File "/Library/Python/2.6/site-packages/django/shortcuts/__init__.py", line 20, in render_to_response
      return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
    File "/Library/Python/2.6/site-packages/django/template/loader.py", line 103, in render_to_string
      t = get_template(template_name)
    File "/Library/Python/2.6/site-packages/django/template/loader.py", line 81, in get_template
      source, origin = find_template_source(template_name)
    File "/Library/Python/2.6/site-packages/django/template/loader.py", line 74, in find_template_source
      raise TemplateDoesNotExist, name
  TemplateDoesNotExist: registration/password_change_form.html
  ...
  

  In total 10 tests failed. At first I thought this was wrong. There must be something wrong with my Django installation. I consulted the Django documentation on testing Django apps and all I could pick up was that when you run python manage.py test it runs the tests of all your INSTALLED_APPS in the settings.py file.

  After a bit of searching, I saw a ticket was created for this and was closed with a wontfix resolution. It made sense since it is test cases that was not implemented yet. Doh! Test Driven Development.

  I do think it could be useful if they added this to the Django docs as a note for n00bs like me.
  
   
• 

  Call me anal, but I really like Django templates

  Stii 11:58 am on October 2, 2009 | Comments: 9 Permalink | Reply
  Tags: django (4), html, PHP (7), Python, templates (2)

  For all the wrong reasons, but still! I just love the fact that it actually uses a .html extension. Am I weird? I do feel a bit weird that I like something so arb. So the template files use the extension of what it actually contains? Wow, I love that. Revolutionary. Don’t ask me why I just thought of that… I cannot answer that truthfully :P Feels like I’m being anal, since it doesn’t seem to matter to anyone else. Most other web development frameworks uses other file extensions. For example in PHP frameworks, the template files have a .php extension. Must be this glorious Rocktober day in Cape Town that is doing this to me!

  
  
   
• 

  The importance of documentation

  Stii 12:46 am on September 16, 2009 | Comments: 0 Permalink | Reply
  Tags: documentation, Open Source (3), PHP (7), Python

  Very good post on open source documentation via @TheKeyboard » Open Source Is Really About Documentation – Twisted vs. Tornado. Chris Brain’s point in short is:

  I think I can give you an executive summary of this blog post: if the documentation for an open source project sucks, nobody but the most hardcore developers will use it.

  

  That couldn’t be more true! The documentation is just about as important as the application. The problem is that it takes time to maintain. Often we as developers are simply too lazy and “will update the documentation later”. I know I am like that, sadly… Will make it next years new years resolution to change that!

  I do think that there could possibly be better ways to do documentation. Look at PHP

  ‘s documentation. It is fairly simple and concise, but I have often found great value in the user comments and examples which you would find underneath every page. I don’t think it is perfect, but it sure helps.

  User contributions play a big role in Open Source documentation and developers with blogs often write their examples on their blogs. I end up finding them by searching Google for it, but it would sure help much more if those posts were linked to the documentation. Something to consider, I think.
  
   
• 

  The Zen of Python

  Stii 2:06 pm on August 17, 2009 | Comments: 0 Permalink | Reply
  Tags: debugging (2), programming (6), Python, Software Dev

  Found at this Stack overflow gem.

  “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”

  :)
  
   
• 

  HTTP conditional GET with Python urllib2

  Stii 10:47 am on July 2, 2009 | Comments: 1 Permalink | Reply
  Tags: Afrigator (23), conditional get, ETag, http, Last-Modified-Date, Python, rss (2), urllib2

  When aggregating or reading crap loads of RSS feeds, it makes little or no sense to read every feed every time you check, when most feeds is updated only once a day. To give you and idea, at Afrigator the size of the feeds are half a gig (500 MB), so if you do that every hour you consume 12 giga bytes of data in 24 hours. This simply to get about 2000 new blog posts per day.

  To alleviate load off the system and data transfers, you can do a HTTP conditional GET which basically check the RSS feed’s HTTP headers to see whether or not the feed was updated since the last time you checked and if it was, you’ll process the feed, else just ignore it. It does this by checking the ETag and Last-Modified-Date HTTP header attributes. It also only fetches the headers and not the entire feed, so only a fraction of the data is retrieved.

  ...
  req = urllib2.Request(url)
  
  req.add_header("If-None-Match", etag)
  req.add_header("If-Modified-Since", lastmodified)
  
  opener = urllib2.build_opener(NotModifiedHandler())
  url_handle = opener.open(req)
  
  if hasattr(url_handle, 'code') and url_handle.code == 304:
      return
  else:
      headers = url_handle.info()
      new_etag = headers.getheader("ETag")
      new_last_modified = headers.getheader("Last-Modified")
  
      if new_etag != None and new_last_modified != None:
          store_new_etag(new_etag, new_last_modified, self.id)
  
      #get the content and write to file
      content = url_handle.read()
  ...
  

  If you’re interested to know the more technical aspects of what happens, see this brilliant post. If you plan to build a feed reader at all, you need to use this function. You will not only kill your bandwidth, but everybody else’s if you don’t use it. If you built your own blogging platform, you need to make sure that you add the necessary ETag and Last-Modified-Date headers to your RSS feed. Will tell you next time how to do that. If you are on WordPress, Blogger or Movable Type it should be fine.
  
   
• 

  Django with apache and mod_wsgi issues

  Stii 8:47 pm on April 14, 2009 | Comments: 1 Permalink | Reply
  Tags: apache (2), django (4), mod_wsgi, Python

  I ran into a fairly common error while setting up Django with mod_wsgi and Apache on Debian. What irritates me about it is that I didn’t pick it up right away, but I’ll blame the long Easter weekend for that. :P

  

  ImportError: Could not import settings 'mysite.settings'
  (Is it on sys.path? Does it have syntax errors?):
  No module named mysite.settings
  

  My wsgi file looked like this:

  import os, sys
  sys.path.append('/home/djangoprojects/mysite/')
  os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
  
  import django.core.handlers.wsgi
  
  _application = django.core.handlers.wsgi.WSGIHandler()
  
  def application(environ, start_response):
      if environ['wsgi.url_scheme'] == 'https':
          environ['HTTPS'] = 'on'
      return _application(environ, start_response)
  

  Like an idiot, I was looking everywhere for the issue, accept the glaring obvious reason. I checked permissions, created a .pth file for the project in the /site-packages/ directory in the Python library. Trust me, I did everything, but the obvious thing.

  See, here is that obvious reason:

  I append the directory “/home/djangoprojects/mysite” to my sys.path which is wrong. Python then looks for the module “mysite” in the directory /home/djangoprojects/mysite which in fact doesn’t exist. Obvious, no?

  To fix this problem, you need to append the directory “/home/djangoprojects/” to your sys.path and it will find the module mysite.settings no problem and it will run smooth.

  Long weekend, surf on the brain. That’s what is wrong!
  
   
• 

  Switched to Python Fabric

  Stii 11:18 am on March 4, 2009 | Comments: 13 Permalink | Reply
  Tags: Afrigator (23), capistrano (3), deployment, fabric (3), Python, svn (4)

  We’ve been using Capistrano to deploy Afrigator to our various servers for a while now. I can seriously not complain or say anything bad about Capistrano. Thing is, I’m more familiar with Python than I am with Ruby, thus for me it just makes more sense for me to use Fabric.

  Here is a small example of how to write a typical deploy script with Python Fabric:

  First, define your various servers which you need to deploy to. Ideally, you’ll have a test, staging and live server. Thus you’ll set them up as follows.

  def test():
      config.fab_user = 'test_user_name'
      config.fab_hosts = ['test.yourserver.com']
  
  def staging():
      config.fab_user = 'staging_user_name'
      config.fab_hosts = ['staging.yourserver.com']
  
  def live():
      config.fab_user = 'live_user_name'
      config.fab_hosts = ['www1.yourserver.com', \
  'www2.yourserver.com', 'www3.yourserver.com']
  

  This allows you to deploy your code to the various servers. Please note, you need to setup automatic login for your different servers. See here how to do automatic logins.

  Next, write the steps you would take to deploy your site manually:

  def deploy():
      "Deploy code to servers"
      msg = "deploying"
      require('fab_hosts', provided_by = [test,staging,live])
      local('svn ci -m "$(msg)"')
      run('svn export repos /path/to/repository/export/')
      run('cp -R /path/to/repository/export/* /path/to/your/site/')
  

  To deploy to the staging server you can run the following command:

  $ fab staging let:msg="Reason for check in" deploy
  

  To deploy to the 3 live servers, all you do is:

  $ fab live let:msg="Reason for check in" deploy
  

  The command works as follows: fab is the command. live/stating/test are the environments you would like to load. If you said staging, it will do the commands in deploy for the staging server. The last part is the command you want to run. In this instance, deploy.

  The let:msg=”Reason for check in” is the coolest bit! It basically allow you to override Fabric variables. If you look closely, in my deploy script I’ve set a variable msg to just say “deploy”. Now that is a stupid SVN message for a commit. In order to commit with meaningful messages, I override the msg variable with my own message. Simple, yet very effective!

  You may want to write a number of different functions in a single fabfile. If you have 100 commands and you’re not one hundred percent sure, just do a:

  $ fab list
  

  and all the commands available to you will be printed with their description. Fabric rocks! Seriously.
  
   
• 

  How to calculate the days between two dates using Python. Quickly.

  Stii 12:56 pm on February 6, 2009 | Comments: 3 Permalink | Reply
  Tags: bash (4), Python, softwaredevelopment (2), tools (3)

  This is one of those pretty useless things, but you never know when you might just have the need for something like this… I have not taken the time to investigate a quicker way using Bash, but my love of Python made me use it without even thinking twice! :)

  I first fired up the interactive Python interpreter by simply typing the command python. Next, I imported the datetime module. Did a simple timedelta between two datetime objects and Jack’s your uncle.

  $ python
  Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
  [GCC 4.0.1 (Apple Inc. build 5465)] on darwin
  Type "help", "copyright", "credits" or "license"
  for more information.
  >>> import datetime
  >>> print (datetime.date(2009, 03, 31) \
  ... - datetime.date(2009, 02, 06)).days
  53
  >>>
  

  Python not only is an excellent programming language, it is also a brilliant general purpose toolbox! Bash/Awk/whatever experts, is there a quicker way maybe? Would love to know.