<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	>

<channel>
	<title>chrisspenblog</title>
	<atom:link href="http://www.chrisspen.com/blog/feed" rel="self" type="application/rss+xml" />
	<link>http://www.chrisspen.com/blog</link>
	<description>knowledge through programming</description>
	<pubDate>Sat, 28 Jan 2012 11:53:31 +0000</pubDate>
	<generator>http://wordpress.org/?v=2.5.1</generator>
	<language>en</language>
			<item>
		<title>Online Variance Calculation in Python</title>
		<link>http://www.chrisspen.com/blog/online-variance-calculation-in-python.html</link>
		<comments>http://www.chrisspen.com/blog/online-variance-calculation-in-python.html#comments</comments>
		<pubDate>Wed, 25 Jan 2012 05:33:19 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=25</guid>
		<description><![CDATA[Since I couldn&#8217;t easily find this code anywhere, I figured I&#8217;d post it here for quick reference:


&#34;&#34;&#34;
2012.1.25 CKS
Incremental calculation of both the mean and variance.
http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
&#34;&#34;&#34;
import unittest
&#160;
## Dumb slow mean/variance formulas.
&#160;
def mean&#40;seq&#41;:
    return sum&#40;seq&#41;/float&#40;len&#40;seq&#41;&#41;
&#160;
def variance&#40;seq&#41;:
    m = mean&#40;seq&#41;
    return sum&#40;&#40;v-m&#41;**2 for v in seq&#41;/float&#40;len&#40;seq&#41;&#41;
&#160;
## Incremental mean/variance formulas.
&#160;
class [...]]]></description>
			<content:encoded><![CDATA[<p>Since I couldn&#8217;t easily find this code anywhere, I figured I&#8217;d post it here for quick reference:</p>
<p><br/></p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #483d8b;">&quot;&quot;&quot;
2012.1.25 CKS
Incremental calculation of both the mean and variance.
http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
&quot;&quot;&quot;</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">unittest</span>
&nbsp;
<span style="color: #808080; font-style: italic;">## Dumb slow mean/variance formulas.</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> mean<span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span>:
    <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">sum</span><span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span>/<span style="color: #008000;">float</span><span style="color: black;">&#40;</span><span style="color: #008000;">len</span><span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">def</span> variance<span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span>:
    m = mean<span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">sum</span><span style="color: black;">&#40;</span><span style="color: black;">&#40;</span>v-m<span style="color: black;">&#41;</span><span style="color: #66cc66;">**</span><span style="color: #ff4500;">2</span> <span style="color: #ff7700;font-weight:bold;">for</span> v <span style="color: #ff7700;font-weight:bold;">in</span> seq<span style="color: black;">&#41;</span>/<span style="color: #008000;">float</span><span style="color: black;">&#40;</span><span style="color: #008000;">len</span><span style="color: black;">&#40;</span>seq<span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #808080; font-style: italic;">## Incremental mean/variance formulas.</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> Stat<span style="color: black;">&#40;</span><span style="color: #008000;">object</span><span style="color: black;">&#41;</span>:
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__init__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
        <span style="color: #008000;">self</span>.<span style="color: black;">mean_sum</span> = <span style="color: #ff4500;">0</span>
        <span style="color: #008000;">self</span>.<span style="color: black;">mean_count</span> = <span style="color: #ff4500;">0</span>
        <span style="color: #008000;">self</span>.<span style="color: black;">last_variance</span> = <span style="color: #ff4500;">0</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #0000cd;">__iadd__</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, value<span style="color: black;">&#41;</span>:
        last_mean = <span style="color: #008000;">self</span>.<span style="color: black;">mean</span>
        <span style="color: #008000;">self</span>.<span style="color: black;">mean_sum</span> += value
        <span style="color: #008000;">self</span>.<span style="color: black;">mean_count</span> += <span style="color: #ff4500;">1</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> last_mean <span style="color: #ff7700;font-weight:bold;">is</span> <span style="color: #ff7700;font-weight:bold;">not</span> <span style="color: #008000;">None</span>:
            <span style="color: #008000;">self</span>.<span style="color: black;">last_variance</span> = <span style="color: #008000;">self</span>.<span style="color: black;">last_variance</span> + <span style="color: black;">&#40;</span>value  - last_mean<span style="color: black;">&#41;</span><span style="color: #66cc66;">*</span><span style="color: black;">&#40;</span>value - <span style="color: #008000;">self</span>.<span style="color: black;">mean</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span>
&nbsp;
    @<span style="color: #008000;">property</span>
    <span style="color: #ff7700;font-weight:bold;">def</span> mean<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">self</span>.<span style="color: black;">mean_count</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span>.<span style="color: black;">mean_sum</span>/<span style="color: #008000;">float</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">mean_count</span><span style="color: black;">&#41;</span>
&nbsp;
    @<span style="color: #008000;">property</span>
    <span style="color: #ff7700;font-weight:bold;">def</span> variance<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">self</span>.<span style="color: black;">last_variance</span>/<span style="color: #008000;">float</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">mean_count</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> Test<span style="color: black;">&#40;</span><span style="color: #dc143c;">unittest</span>.<span style="color: black;">TestCase</span><span style="color: black;">&#41;</span>:
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> <span style="color: #dc143c;">test</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
        nums = <span style="color: #008000;">range</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">1</span>,<span style="color: #ff4500;">7</span><span style="color: black;">&#41;</span>
        s = Stat<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> n <span style="color: #ff7700;font-weight:bold;">in</span> nums:
            s += n
            <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">'mean:'</span>,s.<span style="color: black;">mean</span>
            <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">'variance:'</span>,s.<span style="color: black;">variance</span>
        <span style="color: #008000;">self</span>.<span style="color: black;">assertAlmostEqual</span><span style="color: black;">&#40;</span>s.<span style="color: black;">mean</span>, mean<span style="color: black;">&#40;</span>nums<span style="color: black;">&#41;</span>, <span style="color: #ff4500;">1</span><span style="color: black;">&#41;</span>
        <span style="color: #008000;">self</span>.<span style="color: black;">assertAlmostEqual</span><span style="color: black;">&#40;</span>s.<span style="color: black;">variance</span>, variance<span style="color: black;">&#40;</span>nums<span style="color: black;">&#41;</span>, <span style="color: #ff4500;">2</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">if</span> __name__ == <span style="color: #483d8b;">'__main__'</span>:
    <span style="color: #dc143c;">unittest</span>.<span style="color: black;">main</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/online-variance-calculation-in-python.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Persisting Django&#8217;s Test Database</title>
		<link>http://www.chrisspen.com/blog/persisting-djangos-test-database.html</link>
		<comments>http://www.chrisspen.com/blog/persisting-djangos-test-database.html#comments</comments>
		<pubDate>Sat, 27 Aug 2011 23:26:34 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Django]]></category>

		<category><![CDATA[Python]]></category>

		<category><![CDATA[Uncategorized]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=23</guid>
		<description><![CDATA[After running Django unittests, it may sometimes be useful to manually inspect the test database after unittests are complete. I was working on a complicated network model, trying to resolve an elusive bug in a unittest, and in this specific instance, I thought running a manually written SQL query would be a bit more helpful [...]]]></description>
			<content:encoded><![CDATA[<p>After running Django unittests, it may sometimes be useful to manually inspect the test database after unittests are complete. I was working on a complicated network model, trying to resolve an elusive bug in a unittest, and in this specific instance, I thought running a manually written SQL query would be a bit more helpful than dealing with Django&#8217;s ORM. Since most people don&#8217;t really care about the test database, Django&#8217;s unittest framework is understandably hard-coded to destroy it after the tests have ran. Although there&#8217;s no simply flag to disable this, after digging around in the code, I found it&#8217;s relatively easy enough to change with a couple class method overrides.</p>
<p>First, for convenience, I define a PERSIST_TEST_DATABASE = 1 flag in my settings.py.</p>
<p>Then, my tests.py takes the general form:<br />
<br/></p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">conf</span> <span style="color: #ff7700;font-weight:bold;">import</span> settings
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: #dc143c;">test</span> <span style="color: #ff7700;font-weight:bold;">import</span> TestCase
&nbsp;
<span style="color: #ff7700;font-weight:bold;">if</span> settings.<span style="color: black;">PERSIST_TEST_DATABASE</span>:
    <span style="color: #808080; font-style: italic;"># Disable destruction of the test database.</span>
    <span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">db</span> <span style="color: #ff7700;font-weight:bold;">import</span> transaction, connection, connections, DEFAULT_DB_ALIAS
    <span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: #dc143c;">test</span>.<span style="color: black;">testcases</span> <span style="color: #ff7700;font-weight:bold;">import</span> restore_transaction_methods, connections_support_transactions
    connection.<span style="color: black;">creation</span>.<span style="color: black;">destroy_test_db</span> = <span style="color: #ff7700;font-weight:bold;">lambda</span> <span style="color: #66cc66;">*</span>args, <span style="color: #66cc66;">**</span>kwargs: <span style="color: #008000;">None</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> Test<span style="color: black;">&#40;</span>TestCase<span style="color: black;">&#41;</span>:
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> _fixture_teardown<span style="color: black;">&#40;</span><span style="color: #008000;">self</span><span style="color: black;">&#41;</span>:
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> settings.<span style="color: black;">PERSIST_TEST_DATABASE</span> <span style="color: #ff7700;font-weight:bold;">or</span> <span style="color: #ff7700;font-weight:bold;">not</span> connections_support_transactions<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>:
            <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">super</span><span style="color: black;">&#40;</span>TestCase, <span style="color: #008000;">self</span><span style="color: black;">&#41;</span>._fixture_teardown<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
        <span style="color: #808080; font-style: italic;"># If the test case has a multi_db=True flag, teardown all databases.</span>
        <span style="color: #808080; font-style: italic;"># Otherwise, just teardown default.</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #008000;">getattr</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, <span style="color: #483d8b;">'multi_db'</span>, <span style="color: #008000;">False</span><span style="color: black;">&#41;</span>:
            databases = connections
        <span style="color: #ff7700;font-weight:bold;">else</span>:
            databases = <span style="color: black;">&#91;</span>DEFAULT_DB_ALIAS<span style="color: black;">&#93;</span>
&nbsp;
        restore_transaction_methods<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> db <span style="color: #ff7700;font-weight:bold;">in</span> databases:
            transaction.<span style="color: black;">commit</span><span style="color: black;">&#40;</span>using=db<span style="color: black;">&#41;</span>
            transaction.<span style="color: black;">leave_transaction_management</span><span style="color: black;">&#40;</span>using=db<span style="color: black;">&#41;</span></pre></div></div>

<p>My two overrides are connection.creation.destroy_test_db, which I completely disable, and TestCase._fixture_teardown, which is essentially the same as the original, but tweaked to commit instead of rollback the transaction.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/persisting-djangos-test-database.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Passwd: Authentication token manipulation error</title>
		<link>http://www.chrisspen.com/blog/passwd-authentication-token-manipulation-error.html</link>
		<comments>http://www.chrisspen.com/blog/passwd-authentication-token-manipulation-error.html#comments</comments>
		<pubDate>Wed, 01 Jun 2011 00:00:06 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Linux]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=22</guid>
		<description><![CDATA[I was doing some work with an inherited Kerberos setup, when I noticed my Kerberos-based SVN login suddenly stopped working. Attempting to ssh into the machine revealed the cause:

WARNING: Your password has expired.
You must change your password now and login again!
Fair enough. I personally find password expiration a huge annoyance that provides no real security, [...]]]></description>
			<content:encoded><![CDATA[<p>I was doing some work with an inherited Kerberos setup, when I noticed my Kerberos-based SVN login suddenly stopped working. Attempting to ssh into the machine revealed the cause:<br />
<br/></p>
<pre>WARNING: Your password has expired.
You must change your password now and login again!</pre>
<p>Fair enough. I personally find password expiration a huge annoyance that <a href="http://www.cryptosmith.com/node/218">provides no real security</a>, but sure, why not. So I proceed with the default prompts, only to encounter:<br />
<br/></p>
<pre>Changing password for user myuser.
Kerberos 5 Password:
New password:
Retype new password:
passwd: Authentication token manipulation error</pre>
<p>Fortunately (or unfortunately), this is a common if unhelpfully ambiguous error. <a href="http://mohammednv.wordpress.com/2008/01/08/authentication-token-manipulation-error-when-changing-user-passwords-in-linux/">Several</a> <a href="http://www.ideaexcursion.com/2009/09/11/fixing-authentication-token-manipulation-error-when-changing-passwords-with-passwd/">people</a> <a href="http://fusiontech.wordpress.com/2009/12/10/fixing-linux-problempasswd-authentication-token-manipulation-error/">have</a> posted theories for the underlying cause as well as fixes. Of course, none of the solutions worked for me. Having done a little maintenance work on the system before, I had root access to the machine, so one solution I found was to login as root and run `kpasswd myuser`. Apparently, even though it&#8217;s Kerberos detecting that my password has expired, it was using the default Linux `passwd` tool, and not the Kerberos-equivalent `kpasswd`.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/passwd-authentication-token-manipulation-error.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Restricting Write Access to Django Admin</title>
		<link>http://www.chrisspen.com/blog/restricting-write-access-to-django-admin.html</link>
		<comments>http://www.chrisspen.com/blog/restricting-write-access-to-django-admin.html#comments</comments>
		<pubDate>Thu, 19 May 2011 22:36:48 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Django]]></category>

		<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=21</guid>
		<description><![CDATA[Django has a great auto-generated administration interface. It&#8217;s saved me countless hours of development time and makes basic data maintenance a breeze. Although it&#8217;s designed to be extensible, it&#8217;s not without it&#8217;s quirks and hurtles. Fortunately, most of these can be overcome with a little code inspection. One of these quirks is the superficial inability [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.djangoproject.com/">Django</a> has a great auto-generated administration interface. It&#8217;s saved me countless hours of development time and makes basic data maintenance a breeze. Although it&#8217;s designed to be <a href="http://www.djangobook.com/en/1.0/chapter17/">extensible</a>, it&#8217;s not without it&#8217;s quirks and hurtles. Fortunately, most of these can be overcome with a little code inspection. One of these quirks is the superficial inability to make certain models read-only for specific user groups.</p>
<p>To understand why this may not make sense for certain applications, we have to look at Django&#8217;s two basic permissions, &#8220;staff&#8221; permission, which allows a user to login to admin, but not necessarily do anything else, and &#8220;superuser&#8221; permission, which gives the user access to all Django models.</p>
<p>For one application I worked on, staff users needed to be able to login to admin and read and possibly edit User records. Since the User model stores each user&#8217;s permissions, the main problem we immediately run into is that giving a user permission to edit the User model gives them blanket permission to edit anyone&#8217;s permissions. What this means is that a staff user can give themselves superuser permission, remove another user&#8217;s superuser permission, assign an incorrect group permission, and generally wreak havoc. Now, in keeping with The Zen of Admin, if you&#8217;ve given a user staff permission, then they can probably be trusted not to screw around with these permissions. However, this is essentially an honor system, which may not always give you peace of mind. It&#8217;s still a great temptation and potential opening for accidental misuse.</p>
<p>Fortunately, we can fix this problem by disallowing staff users from editing user permissions with a relatively simple change to Django&#8217;s UserAdmin. Specifically, we can define a custom UserAdmin, inheriting from the default UserAdmin, by defining the following in &lt;your_app&gt;/admin.py:</p>
<p><br/></p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">contrib</span> <span style="color: #ff7700;font-weight:bold;">import</span> admin
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">contrib</span>.<span style="color: black;">admin</span>.<span style="color: black;">util</span> <span style="color: #ff7700;font-weight:bold;">import</span> flatten_fieldsets
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">contrib</span>.<span style="color: black;">auth</span>.<span style="color: black;">admin</span> <span style="color: #ff7700;font-weight:bold;">import</span> UserAdmin <span style="color: #ff7700;font-weight:bold;">as</span> _UserAdmin
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">core</span>.<span style="color: #dc143c;">exceptions</span> <span style="color: #ff7700;font-weight:bold;">import</span> PermissionDenied
<span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">shortcuts</span> <span style="color: #ff7700;font-weight:bold;">import</span> get_object_or_404
&nbsp;
<span style="color: #ff7700;font-weight:bold;">class</span> UserAdmin<span style="color: black;">&#40;</span>_UserAdmin<span style="color: black;">&#41;</span>:
    _readonly_fields = <span style="color: black;">&#91;</span><span style="color: black;">&#93;</span> <span style="color: #808080; font-style: italic;"># Default fields that are readonly for everyone.</span>
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> get_readonly_fields<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, request, obj<span style="color: black;">&#41;</span>:
        readonly = <span style="color: #008000;">list</span><span style="color: black;">&#40;</span><span style="color: #008000;">self</span>._readonly_fields<span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_staff</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #ff7700;font-weight:bold;">not</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span>:
            <span style="color: #ff7700;font-weight:bold;">if</span> obj.<span style="color: black;">is_superuser</span>:
                <span style="color: #808080; font-style: italic;"># Prevent a staff user from editing anything of a superuser.</span>
                readonly.<span style="color: black;">extend</span><span style="color: black;">&#40;</span>flatten_fieldsets<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">declared_fieldsets</span><span style="color: black;">&#41;</span><span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">else</span>:
                <span style="color: #808080; font-style: italic;"># Prevent a non-superuser from editing sensitive security-related fields.</span>
                readonly.<span style="color: black;">extend</span><span style="color: black;">&#40;</span><span style="color: black;">&#91;</span><span style="color: #483d8b;">'is_staff'</span>, <span style="color: #483d8b;">'is_superuser'</span>, <span style="color: #483d8b;">'user_permissions'</span>, <span style="color: #483d8b;">'groups'</span><span style="color: black;">&#93;</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">return</span> readonly
&nbsp;
    <span style="color: #ff7700;font-weight:bold;">def</span> user_change_password<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>, request, <span style="color: #008000;">id</span><span style="color: black;">&#41;</span>:
        <span style="color: #808080; font-style: italic;"># Disallow a non-superuser from changing the password of a superuser.</span>
        <span style="color: #dc143c;">user</span> = get_object_or_404<span style="color: black;">&#40;</span><span style="color: #008000;">self</span>.<span style="color: black;">model</span>, pk=<span style="color: #008000;">id</span><span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> <span style="color: #ff7700;font-weight:bold;">not</span> request.<span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span> <span style="color: #ff7700;font-weight:bold;">and</span> <span style="color: #dc143c;">user</span>.<span style="color: black;">is_superuser</span>:
            <span style="color: #ff7700;font-weight:bold;">raise</span> PermissionDenied
        <span style="color: #ff7700;font-weight:bold;">return</span> <span style="color: #008000;">super</span><span style="color: black;">&#40;</span>UserAdmin, <span style="color: #008000;">self</span><span style="color: black;">&#41;</span>.<span style="color: black;">user_change_password</span><span style="color: black;">&#40;</span>request, <span style="color: #008000;">id</span><span style="color: black;">&#41;</span>
&nbsp;
admin.<span style="color: #dc143c;">site</span>.<span style="color: black;">unregister</span><span style="color: black;">&#40;</span>User<span style="color: black;">&#41;</span>
admin.<span style="color: #dc143c;">site</span>.<span style="color: black;">register</span><span style="color: black;">&#40;</span>User, UserAdmin<span style="color: black;">&#41;</span></pre></div></div>

<p>This prevents staff users from editing any user permissions, but still allows them to edit other user data, and prevents them from editing anything for a superuser.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/restricting-write-access-to-django-admin.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>How to Extract a Webpage’s Main Article Content: The Unicode Edition</title>
		<link>http://www.chrisspen.com/blog/how-to-extract-a-webpage%e2%80%99s-main-article-content-the-unicode-edition.html</link>
		<comments>http://www.chrisspen.com/blog/how-to-extract-a-webpage%e2%80%99s-main-article-content-the-unicode-edition.html#comments</comments>
		<pubDate>Mon, 03 Jan 2011 23:00:00 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Python]]></category>

		<category><![CDATA[html]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=20</guid>
		<description><![CDATA[When I originally wrote html2text.py, my focus was only on extracting English text from webpages, so I didn&#8217;t give much thought to handling Unicode. Ignoring anything but ASCII would suffice. However, I was recently commissioned to extend the script&#8217;s functionality to use Unicode, so it could extract text in nearly any language found on the [...]]]></description>
			<content:encoded><![CDATA[<p>When I <a href="/blog/how-to-extract-a-webpages-main-article-content.html">originally</a> wrote html2text.py, my focus was only on extracting English text from webpages, so I didn&#8217;t give much thought to handling Unicode. Ignoring anything but ASCII would suffice. However, I was recently commissioned to extend the script&#8217;s functionality to use Unicode, so it could extract text in nearly any language found on the Web. The modifications turned out to be relatively straight-forward, made easier with the use of the handy <a href="http://chardet.feedparser.org/">chardet</a> package to detect encoding. I also cleaned up the command line interface, to make the various parameters more accessible.</p>
<p><del datetime="2011-12-17T18:49:48+00:00">You can download the code <a href="/public/html2text-20110102.zip">here</a>.</del></p>
<p><strong>Update 2011-12-17</strong>: To avoid naming conflicts with similar scripts, I&#8217;ve renamed the script to webarticle2text and <a href="https://github.com/chrisspen/webarticle2text">published it to github.com</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/how-to-extract-a-webpage%e2%80%99s-main-article-content-the-unicode-edition.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>How To Install MBROLA Voices For Use With Festival On Ubuntu</title>
		<link>http://www.chrisspen.com/blog/how-to-install-mbrola-voices-for-use-with-festival-on-ubuntu.html</link>
		<comments>http://www.chrisspen.com/blog/how-to-install-mbrola-voices-for-use-with-festival-on-ubuntu.html#comments</comments>
		<pubDate>Tue, 03 Aug 2010 21:18:05 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Linux]]></category>

		<category><![CDATA[Ubuntu]]></category>

		<category><![CDATA[festival]]></category>

		<category><![CDATA[mbrola]]></category>

		<category><![CDATA[text-to-speech]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=19</guid>
		<description><![CDATA[Festival is a great text-to-speech engine. MBROLA also contains some impressive voice files that have notably higher quality than Festival&#8217;s default voices. Both of these systems are packaged in Ubuntu, and even though Festival can use MBROLA&#8217;s voice files, as of Ubuntu 10.04, there doesn&#8217;t seem to be a package to easily allows you to [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.cstr.ed.ac.uk/projects/festival/">Festival</a> is a great text-to-speech engine. <a href="http://tcts.fpms.ac.be/synthesis/mbrola.html">MBROLA</a> also contains some impressive voice files that have notably higher quality than Festival&#8217;s default voices. Both of these systems are packaged in Ubuntu, and even though Festival can use MBROLA&#8217;s voice files, as of Ubuntu 10.04, there doesn&#8217;t seem to be a package to easily allows you to do this.</p>
<p>There&#8217;s an <a href="http://ubuntuforums.org/showthread.php?t=751169">excellent guide</a> on how to accomplish this manually. However, since it was written, a few more packages are now included in Ubuntu, allowing several steps to be skipped. Since most of the MBROLA voices are available as packages, the only steps needed to use them from Festival are to download a Festival-to-MBROLA wrapper and move the MBROLA voice files into Festival&#8217;s directory.</p>
<p>The following script installs MBROLA&#8217;s male British English voice for use with Festival. All MBROLA voices are represented by a &#8220;two-letter number&#8221; combination (e.g. en1). To install a different voice, change the VOICE variable to suit.<br />
<br/></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
<span style="color: #000000; font-weight: bold;">set</span> <span style="color: #660033;">-e</span>
&nbsp;
<span style="color: #007800;">VOICE</span>=en1
<span style="color: #007800;">VLANG</span>=english
&nbsp;
<span style="color: #666666; font-style: italic;"># Install all required packages.</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> festival mbrola mbrola-<span style="color: #800000;">${VOICE}</span> esound-clients festlex-oald
&nbsp;
<span style="color: #666666; font-style: italic;"># Download wrappers for mbrola voices.</span>
<span style="color: #7a0874; font-weight: bold;">cd</span> ~<span style="color: #000000; font-weight: bold;">/</span>Desktop
<span style="color: #c20cb9; font-weight: bold;">wget</span> http:<span style="color: #000000; font-weight: bold;">//</span>www.cstr.ed.ac.uk<span style="color: #000000; font-weight: bold;">/</span>downloads<span style="color: #000000; font-weight: bold;">/</span>festival<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">1.95</span><span style="color: #000000; font-weight: bold;">/</span>festvox_<span style="color: #800000;">${VOICE}</span>.tar.gz
<span style="color: #c20cb9; font-weight: bold;">tar</span> xvf festvox_<span style="color: #800000;">${VOICE}</span>.tar.gz
<span style="color: #c20cb9; font-weight: bold;">rm</span> <span style="color: #660033;">-f</span> festvox_<span style="color: #800000;">${VOICE}</span>.tar.gz
&nbsp;
<span style="color: #666666; font-style: italic;"># Copy MBROLA voices into Festival's voice directory.</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">mkdir</span> <span style="color: #660033;">-p</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>share<span style="color: #000000; font-weight: bold;">/</span>festival<span style="color: #000000; font-weight: bold;">/</span>voices<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VLANG}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>_mbrola<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">mv</span> festival<span style="color: #000000; font-weight: bold;">/</span>lib<span style="color: #000000; font-weight: bold;">/</span>voices<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VLANG}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>_mbrola<span style="color: #000000; font-weight: bold;">/*</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>share<span style="color: #000000; font-weight: bold;">/</span>festival<span style="color: #000000; font-weight: bold;">/</span>voices<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VLANG}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>_mbrola<span style="color: #000000; font-weight: bold;">/</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">cp</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>share<span style="color: #000000; font-weight: bold;">/</span>mbrola<span style="color: #000000; font-weight: bold;">/</span>voices<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>share<span style="color: #000000; font-weight: bold;">/</span>festival<span style="color: #000000; font-weight: bold;">/</span>voices<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VLANG}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>_mbrola<span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${VOICE}</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Cleanup.</span>
<span style="color: #c20cb9; font-weight: bold;">rm</span> <span style="color: #660033;">-rf</span> ~<span style="color: #000000; font-weight: bold;">/</span>Desktop<span style="color: #000000; font-weight: bold;">/</span>festival
&nbsp;
<span style="color: #666666; font-style: italic;"># Test.</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'(voice_'</span><span style="color: #800000;">${VOICE}</span><span style="color: #ff0000;">'_mbrola)(SayText &quot;Installation complete!&quot;)'</span> <span style="color: #000000; font-weight: bold;">|</span> esddsp festival <span style="color: #660033;">--pipe</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/how-to-install-mbrola-voices-for-use-with-festival-on-ubuntu.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Ubuntu Flash Drive Encryption Script</title>
		<link>http://www.chrisspen.com/blog/ubuntu-flash-drive-encryption-script.html</link>
		<comments>http://www.chrisspen.com/blog/ubuntu-flash-drive-encryption-script.html#comments</comments>
		<pubDate>Thu, 08 Jul 2010 22:28:44 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Linux]]></category>

		<category><![CDATA[Ubuntu]]></category>

		<category><![CDATA[encryption]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=18</guid>
		<description><![CDATA[I was researching methods to encrypt USB flash drives, to protect personal data in the event they&#8217;re lost or stolen, and I found a few good write-ups on the subject using tools available in Ubuntu. Although the current distribution of Gnome includes pretty good support for auto-detecting and prompting for the password for encrypted drives, [...]]]></description>
			<content:encoded><![CDATA[<p>I was researching methods to encrypt USB flash drives, to protect personal data in the event they&#8217;re lost or stolen, and I found a few <a href="http://www.emcken.dk/weblog/archives/164-encrypted-usb-drive-in-ubuntu.html">good</a> <a href="https://help.ubuntu.com/community/EncryptedFilesystemHowto">write-ups</a> on the subject using tools available in <a href="http://www.ubuntu.com/">Ubuntu</a>. Although the current distribution of Gnome includes pretty good support for auto-detecting and prompting for the password for encrypted drives, I couldn&#8217;t find any completely baked tools for encrypting the drives. So I wrote a simple script to streamline the process.<br />
<br/></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
<span style="color: #000000; font-weight: bold;">set</span> <span style="color: #660033;">-e</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Drive Encrypter&quot;</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Determine the device name.</span>
<span style="color: #c20cb9; font-weight: bold;">mount</span> <span style="color: #666666; font-style: italic;"># Display all mounted devices to help the user find the device name.</span>
<span style="color: #c20cb9; font-weight: bold;">read</span> <span style="color: #660033;">-p</span> <span style="color: #ff0000;">&quot;What is device name (e.g. /dev/&lt;devicename&gt;, from the mount list above)? &quot;</span> <span style="color: #660033;">-e</span> DEVICE
&nbsp;
<span style="color: #666666; font-style: italic;"># Verify all required packages are installed.</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Checking packages...&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> cryptsetup
<span style="color: #c20cb9; font-weight: bold;">sudo</span> modprobe dm-crypt
&nbsp;
<span style="color: #666666; font-style: italic;"># Overwrite with random data in order to slow down an attack on the encryption.</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Erasing device...&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">umount</span> <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span><span style="color: #007800;">$DEVICE</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> badblocks <span style="color: #660033;">-c</span> <span style="color: #000000;">10240</span> <span style="color: #660033;">-s</span> <span style="color: #660033;">-w</span> <span style="color: #660033;">-t</span> random <span style="color: #660033;">-v</span> <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span><span style="color: #007800;">$DEVICE</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Creating encrypted partition...&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> cryptsetup <span style="color: #660033;">--verbose</span> <span style="color: #660033;">--verify-passphrase</span> luksFormat <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span><span style="color: #007800;">$DEVICE</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> cryptsetup luksOpen <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span><span style="color: #007800;">$DEVICE</span> <span style="color: #007800;">$DEVICE</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> mkfs.ext3 <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span>mapper<span style="color: #000000; font-weight: bold;">/</span><span style="color: #007800;">$DEVICE</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> cryptsetup luksClose <span style="color: #007800;">$DEVICE</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Done.&quot;</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Remove the drive and then plug it back in, and you should be prompted for a password to access the device.&quot;</span>
<span style="color: #666666; font-style: italic;">#Note, you may have to run this to access the device from a normal account:</span>
<span style="color: #666666; font-style: italic;">#sudo chown -R user:user /media/mountpoint</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/ubuntu-flash-drive-encryption-script.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>How To Remotely Backup A Web Server With No Install</title>
		<link>http://www.chrisspen.com/blog/how-to-remotely-backup-a-web-server-with-no-install.html</link>
		<comments>http://www.chrisspen.com/blog/how-to-remotely-backup-a-web-server-with-no-install.html#comments</comments>
		<pubDate>Wed, 31 Mar 2010 22:52:07 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Apache]]></category>

		<category><![CDATA[Linux]]></category>

		<category><![CDATA[MySQL]]></category>

		<category><![CDATA[PostgreSQL]]></category>

		<category><![CDATA[backup]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=17</guid>
		<description><![CDATA[I was researching simple methods for backing up a basic Linux web server, and I found an excellent script that handles files and databases, and also dumps everything to a remote server. I was looking for something a bit less complicated and overkill than Bacula, but more comprehensive than simply archiving the filesystem, and this [...]]]></description>
			<content:encoded><![CDATA[<p>I was researching simple methods for backing up a basic Linux web server, and I found an <a href="http://www.cyberciti.biz/faq/how-to-back-up-a-web-server/">excellent</a> script that handles files and databases, and also dumps everything to a remote server. I was looking for something a bit less complicated and overkill than <a href="http://www.bacula.org/">Bacula</a>, but more comprehensive than simply archiving the filesystem, and this fills the niche nicely.</p>
<p>Taking this solution one step further, I simplified the install requirements a bit by rewriting the script to run the backup remotely by logging in via SSH and pulling the data directly from the source machine. Instead of having to install the script and cron job on each machine you want to backup, you just install it once on the destination server, and ensure it has SSH access to each target. I also modified it to record a list of all installed packages, which would be useful if the system had to be reconstructed. Intuitively, it seems easier to deal with a single file, so I also wrap all backup parts into a single archive.</p>
<p>To start, you&#8217;ll want to ensure the script&#8217;s user has SSH access to the target machine by locally running something like:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">ssh-copy-id tunneluser<span style="color: #000000; font-weight: bold;">@</span>12.345.678.90</pre></div></div>

<p>You&#8217;ll also want to ensure it has permission to access all the data it needs to backup. On the target machine, you&#8217;d run something like:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>sbin<span style="color: #000000; font-weight: bold;">/</span>useradd tunneluser
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>sbin<span style="color: #000000; font-weight: bold;">/</span>usermod <span style="color: #660033;">-a</span> <span style="color: #660033;">-G</span> apache tunneluser
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">passwd</span> tunneluser
<span style="color: #c20cb9; font-weight: bold;">sudo</span> mysql <span style="color: #660033;">-e</span> <span style="color: #ff0000;">&quot;GRANT USAGEG ON *.* TO tunneluser@localhost IDENTIFIED BY 'mysqlpassword';GRANT ALL PRIVILEGES ON *.* TO tunneluser@localhost;&quot;</span></pre></div></div>

<p>Finally, the script itself:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
<span style="color: #666666; font-style: italic;"># Simple script for remotely backing up all data on a webserver.</span>
<span style="color: #666666; font-style: italic;"># Adapted from http://www.cyberciti.biz/faq/how-to-back-up-a-web-server/</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Define remote login.</span>
<span style="color: #007800;">SSHUSER</span>=<span style="color: #ff0000;">&quot;tunneluser&quot;</span>
<span style="color: #007800;">SSHHOST</span>=<span style="color: #ff0000;">&quot;12.345.678.90&quot;</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Define remote and local backup destinations.</span>
<span style="color: #007800;">LOCAL_BACKUP_ROOT</span>=<span style="color: #ff0000;">&quot;/var/backups&quot;</span>
<span style="color: #007800;">NOW</span>=$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #ff0000;">&quot;%F&quot;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>
<span style="color: #007800;">BACKUP_ROOT</span>=<span style="color: #ff0000;">&quot;/tmp/backup/<span style="color: #007800;">$NOW</span>&quot;</span>
<span style="color: #007800;">BACKUP_FILENAME</span>=<span style="color: #ff0000;">&quot;backup-<span style="color: #007800;">$SSHHOST</span>-<span style="color: #007800;">$NOW</span>.tar.gz&quot;</span>
<span style="color: #007800;">BACKUP_PATH</span>=<span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_ROOT</span>/<span style="color: #007800;">$BACKUP_FILENAME</span>&quot;</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Run a script on the remote host.</span>
<span style="color: #c20cb9; font-weight: bold;">ssh</span> <span style="color: #660033;">-T</span> <span style="color: #007800;">$SSHUSER</span><span style="color: #000000; font-weight: bold;">@</span><span style="color: #007800;">$SSHHOST</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;EOI
&nbsp;
TERM=dumb
export TERM
&nbsp;
# File directories to backup.
DIRS=&quot;/var/www/html/ /etc&quot;
&nbsp;
# Paths for binary files
TAR=&quot;/bin/tar&quot;
PGDUMP=&quot;/usr/bin/pg_dump&quot;
MYSQLDUMP=&quot;/usr/bin/mysqldump&quot;
GZIP=&quot;/bin/gzip&quot;
LOGGER=&quot;/usr/bin/logger&quot;
&nbsp;
# Set Pgsql username
PGSQLUSER=&quot;tunneluser&quot;
&nbsp;
# Set MySQL username and password
MYSQLUSER=&quot;tunneluser&quot;
MYSQLPASSWORD=&quot;mysqlpassword&quot;
&nbsp;
# Backup file name hostname.time.tar.gz
BFILE=&quot;web.tar.gz&quot;
PFILE=&quot;pg.sql.gz&quot;
MFILE=&quot;mysql.sq.gz&quot;
PKGFILE=&quot;installed-software.log&quot;
&nbsp;
# Store todays date.
NOW=\$(date +&quot;%F&quot;)
echo \$NOW
&nbsp;
# Store backup path.
BACKUP_ROOT=&quot;$BACKUP_ROOT&quot;
[ ! -d \$BACKUP_ROOT ] &amp;&amp; mkdir -p \${BACKUP_ROOT}
BACKUP_FILENAME=&quot;$BACKUP_FILENAME&quot;
&nbsp;
# Log backup start time in /var/log/messages
\$LOGGER &quot;*** Backup started @ \$(date) ***&quot;
&nbsp;
# Backup package list.
echo &quot;Backing up package list...&quot;
rpm -qa &gt; \${BACKUP}/\${PKGFILE} # Redhat
#dpkg --get-selections &gt; \${BACKUP}/\${PKGFILE} # Debian
&nbsp;
# Backup webserver directories.
echo &quot;Backing up directories...&quot;
\$TAR -zcf \${BACKUP_ROOT}/\${BFILE} \${DIRS}
&nbsp;
# Backup PgSQL.
echo &quot;Backing up PgSQL...&quot;
\$PGDUMP -x -D -U\${PGSQLUSER} | \$GZIP -c &gt; \${BACKUP_ROOT}/\${PFILE}
&nbsp;
# Backup MySQL.
echo &quot;Backing up MySQL...&quot;
\$MYSQLDUMP -u \${MYSQLUSER} -h localhost -p\${MYSQLPASSWORD} --all-databases --lock-all-tables | \$GZIP -9 &gt; \${BACKUP_ROOT}/\${MFILE}
&nbsp;
# Create final file and cleanup.
echo &quot;Cleaning up...&quot;
\$TAR -zcf \${BACKUP_FILENAME} \${BACKUP_ROOT}/\${BFILE} \${BACKUP_ROOT}/\${MFILE} \${BACKUP_ROOT}/\${PKGFILE} \${BACKUP_ROOT}/\${PFILE}
rm -Rf \${BACKUP_ROOT}
&nbsp;
# Log backup end time in /var/log/messages
\$LOGGER &quot;*** Backup Ended @ \$(date) ***&quot;
&nbsp;
exit
EOI</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Downloading archive...&quot;</span>
<span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #000000; font-weight: bold;">!</span> <span style="color: #660033;">-d</span> <span style="color: #007800;">$LOCAL_BACKUP_ROOT</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span> <span style="color: #000000; font-weight: bold;">&amp;&amp;</span> <span style="color: #c20cb9; font-weight: bold;">mkdir</span> <span style="color: #660033;">-p</span> <span style="color: #800000;">${LOCAL_BACKUP_ROOT}</span>
<span style="color: #c20cb9; font-weight: bold;">scp</span> <span style="color: #660033;">-rp</span> <span style="color: #007800;">$SSHUSER</span><span style="color: #000000; font-weight: bold;">@</span><span style="color: #007800;">$SSHHOST</span>:<span style="color: #007800;">$BACKUP_PATH</span> <span style="color: #800000;">${LOCAL_BACKUP_ROOT}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${BACKUP_FILENAME}</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Backup complete.&quot;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/how-to-remotely-backup-a-web-server-with-no-install.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Implementing a Simple Machine Learning Intelligence to Play Rock-Paper-Scissors in Javascript</title>
		<link>http://www.chrisspen.com/blog/implementing-a-simple-machine-learning-intelligence-to-play-rock-paper-scissors-in-javascript.html</link>
		<comments>http://www.chrisspen.com/blog/implementing-a-simple-machine-learning-intelligence-to-play-rock-paper-scissors-in-javascript.html#comments</comments>
		<pubDate>Wed, 02 Dec 2009 04:42:45 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Javascript]]></category>

		<category><![CDATA[machine learning]]></category>

		<category><![CDATA[rock paper scissors]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=15</guid>
		<description><![CDATA[Rock-paper-scissors is not a complicated game. However, there are some that not only enjoy the game, but take pride in professing skill at it. But is it possible to have skill at rock-paper-scissors?
In the real world, where a human plays against a human, &#8220;cheating&#8221; is relatively easy. A player can gain an advantage by subtly [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://en.wikipedia.org/wiki/Rock-paper-scissors">Rock-paper-scissors</a> is not a complicated game. However, there are some that not only <a href="http://www.worldrps.com/">enjoy the game</a>, but take pride in <a href="http://www.usarps.com/">professing skill</a> at it. But is it possible to have skill at rock-paper-scissors?</p>
<p>In the real world, where a human plays against a human, &#8220;cheating&#8221; is relatively easy. A player can gain an advantage by subtly delaying their release just long enough to read their opponent&#8217;s body language and choose a winning gesture. For sake of argument, let&#8217;s ignore this and assume the players play fairly, where each is completely unaware of their opponent&#8217;s move.</p>
<p>In this case, optimal play between two players making truly random gestures ends in a tie (on average). The only way to repeatedly win is to somehow predict your opponent&#8217;s gesture, which is impossible if your opponent is truly random. Only if your opponent has a bias can statistics be used to potentially model and predict their gesture.</p>
<p>To demonstrate this, I implemented three simple prediction routines.</p>
<p>To play a round against each routine, just click a link representing the gesture you&#8217;d like to make. The routine will then calculate its own gesture (not looking at your current choice of course), and then display the result.<br />
<script type="text/javascript" src="/ropasc/jquery-1.3.2.min.js"></script><br />
<script type="text/javascript" src="/ropasc/ropasc.js"></script></p>
<link media="screen" type="text/css" href="/ropasc/ropasc.css" rel="stylesheet" />
<p>1. Conditional-0<br />
All the opponent&#8217;s past gestures are counted, and these counts are used as weights when pseudo-randomly selecting a gesture. For example, if you&#8217;ve picked rock once, paper four times, and scissors five times, then there will be a 10% chance that the routine will choose paper, 40% for scissors, and 50% for rock.</p>
<p></p>
<div id="frequency-brain"></div>
<p><script type="text/javascript">
$(document).ready(function(){
    g1 = (new ROPASC.Game('#frequency-brain', ROPASC.FrequencyBrain));
});
</script></p>
<p>2. Conditional-1<br />
This method tracks the last two gestures made by the opponent, and counts the number of each transition. These counts are again used to weight the random selection of a gesture. This allows it to see simple sequential patterns where an opponent&#8217;s next gesture is dependent on their last gesture.</p>
<p></p>
<div id="conditional-1-brain"></div>
<p><script type="text/javascript">
$(document).ready(function(){
    g2 = (new ROPASC.Game('#conditional-1-brain', ROPASC.Conditional1Brain));
});
</script></p>
<p>3. Conditional-2<br />
Similar to the previous method, this one tracks a combination of the opponent&#8217;s last gesture, and its own next-to-last gesture. This allows it to see simple dependence between the opponent&#8217;s gesture and its own.</p>
<p></p>
<div id="conditional-2-brain"></div>
<p><script type="text/javascript">
$(document).ready(function(){
    g3 = (new ROPASC.Game('#conditional-2-brain', ROPASC.Conditional2Brain));
});
</script></p>
<p>It&#8217;s a strange game in that the player with the most entropy wins in the long run. To see the code <a href="/ropasc/ropasc.js">click here</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/implementing-a-simple-machine-learning-intelligence-to-play-rock-paper-scissors-in-javascript.html/feed</wfw:commentRss>
		</item>
		<item>
		<title>Handling PostgreSQL Integrity Errors in Django</title>
		<link>http://www.chrisspen.com/blog/handling-postgresql-integrity-errors-in-django.html</link>
		<comments>http://www.chrisspen.com/blog/handling-postgresql-integrity-errors-in-django.html#comments</comments>
		<pubDate>Fri, 18 Sep 2009 13:00:05 +0000</pubDate>
		<dc:creator>Chris</dc:creator>
		
		<category><![CDATA[Django]]></category>

		<category><![CDATA[MySQL]]></category>

		<category><![CDATA[PostgreSQL]]></category>

		<guid isPermaLink="false">http://www.chrisspen.com/blog/?p=14</guid>
		<description><![CDATA[I have some basic Django code that attempts to insert a new record into a PostgreSQL database. It&#8217;s wrapped in a try/except statement in case a unique constraint is violated, in which case I&#8217;ll typically ignore it and move on.
So the general pattern is:

from django.db import IntegrityError
import models
try:
    myNewRecord = models.MyTable&#40;data=blah&#41;
  [...]]]></description>
			<content:encoded><![CDATA[<p>I have some basic <a href="http://www.djangoproject.com/">Django</a> code that attempts to insert a new record into a <a href="http://www.postgresql.org/">PostgreSQL</a> database. It&#8217;s wrapped in a try/except statement in case a unique constraint is violated, in which case I&#8217;ll typically ignore it and move on.</p>
<p>So the general pattern is:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">db</span> <span style="color: #ff7700;font-weight:bold;">import</span> IntegrityError
<span style="color: #ff7700;font-weight:bold;">import</span> models
<span style="color: #ff7700;font-weight:bold;">try</span>:
    myNewRecord = models.<span style="color: black;">MyTable</span><span style="color: black;">&#40;</span>data=blah<span style="color: black;">&#41;</span>
    myNewRecord.<span style="color: black;">save</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">except</span> IntegrityError:
    <span style="color: #ff7700;font-weight:bold;">pass</span>
&nbsp;
myOtherRecord = models.<span style="color: black;">MyOtherTable</span><span style="color: black;">&#40;</span>data=blah<span style="color: black;">&#41;</span>
<span style="color: #808080; font-style: italic;"># Throws an error if an IntegrityError was previously thrown.</span>
myOtherRecord.<span style="color: black;">save</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span></pre></div></div>

<p>I&#8217;d been previously using <a href="http://www.mysql.com/">MySQL</a>, but after switching to PostgreSQL I started getting a strange error when I tried to do any model access after an IntegrityError had been thrown, even though this access was outside the try/except:</p>
<p>&#8220;ProgrammingError: current transaction is aborted, commands ignored until end of transaction block&#8221;</p>
<p>What I eventually discovered, with a little <a href="http://www.errorhelp.com/search/details/69807/programmingerror-current-transaction-is-aborted-commands-ignored-until-end-of-transaction-block-appears-in-django-when-the-current-session-is-blocked">help</a>, is that Django&#8217;s PostgreSQL wrapper has some unusual transaction handling, so that if an error occurs then the entire connection must be closed to actually close the transaction.</p>
<p>The simple fix was to add connection.close() to the my except clause:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #ff7700;font-weight:bold;">from</span> django.<span style="color: black;">db</span> <span style="color: #ff7700;font-weight:bold;">import</span> IntegrityError, connection
<span style="color: #ff7700;font-weight:bold;">import</span> models
<span style="color: #ff7700;font-weight:bold;">try</span>:
    myNewRecord = models.<span style="color: black;">MyTable</span><span style="color: black;">&#40;</span>data=blah<span style="color: black;">&#41;</span>
    myNewRecord.<span style="color: black;">save</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #ff7700;font-weight:bold;">except</span> IntegrityError:
    <span style="color: #808080; font-style: italic;"># Required to clear PostgreSQL's failed transaction.</span>
    connection.<span style="color: black;">close</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
myOtherRecord = models.<span style="color: black;">MyOtherTable</span><span style="color: black;">&#40;</span>data=blah<span style="color: black;">&#41;</span>
<span style="color: #808080; font-style: italic;"># Now works correctly.</span>
myOtherRecord.<span style="color: black;">save</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.chrisspen.com/blog/handling-postgresql-integrity-errors-in-django.html/feed</wfw:commentRss>
		</item>
	</channel>
</rss>

