Dropbox can't sync because folder is empty.

 I've had this a few times now, and failed to track it down, resorting to things like clearing the .dropbox.cache, quitting dropbox, restarting, disabling selective sync etc.

I thought that the problem was due to a script I run fairly often, which deletes all log files in all subfolders in a project and also pushes changes to boiler plate code to any folder in the project that currently has a copy. I thought that maybe if I ran this too frequently, while a previous sync was in progress, it would just be too much for Dropbox.

What I've just discovered though, is that the blocking folder, which is empty/deleted in Dropbox, still exists on another machine and that a log file in that folder remains open because the Python script that owns it is still running.

I quit the script and all was well once again.

So maybe if you get this error, check that a file isn't open.  If you can't be bothered to track it down, a good old turn-it-off-and-on-again will probably do the trick.

"Pride in an immutable state" OR "I'm not homophobic, but..."

First, let me say the Tim Cook's declaration in Business Week was great. I think it can only do good, it brought a tear to the eye.

But…(here we go!)

I do have an issue with the word 'Pride'.  Note: "I have an issue". It's an opinion, and I may come across as being a homophobic arsehole.  That's not my intent.  I certainly have had to overcome my own own mild socially acquired homophobia, and there may yet be work to be done to completely eradicate it, but I don't consider myself homophobic any more.

My homophobia, which probably involved some teasing and name-calling of Neil Si_______ds in junior school* mostly evaporated when I found out that someone I worked with was gay.  It's quite possible that I'd worked with other gay people and had no idea, of course.  Afterwards, I could perhaps sense that he was more sensitive, more caring and gentle than the average bloke. But if someone found out I was gay - I'm not, but if by some misunderstanding they came to believe that I was gay, they may also suddenly see some signifiers, like being more sensitive, caring and gentle that the average bloke.  Those signifiers are mostly bollocks (figurative bollocks, that is).

Halfway through, I realise this is a bit male-straight/gay.  I'm only speaking as a cis-white-male and I only have a little contact with the LGBTQ world. I'm sorry if there's some other way I'm being a dick that I'm not aware of.

So. Pride.

I could say "Gay? That's nothing to be proud of.", and you may read that as a sneer of disapproval, such is the written word.  So forget that. I'll make a pretentious statement instead:

Take pride in achievements, not in a immutable state of your being. 

 Some examples?

I'm proud to be a woman!

I'm proud to be an American!

I'm a proud Englishman!

You perhaps need to imagine Nigel Farage uttering that last one, and then imagining him saying this instead:

I'm proud that my mother had the presence of mind to be in England when she expelled me from her womb, because that has made me better than Johnny Foreigner through no personal effort whatsoever.

And I kind of lump "I'm proud to be gay" in with that.  It's obviously not as odious as Farage's Nationalism, but thinking about this a bit today eventually led me to the aforementioned pretentious declaration.

At first I was trying to think of better words. 'Unashamed', for example, which, while accurate, is intrinsically rubbish because it also suggest shame in the first place.  I figured the best word was 'Am'. As in "I am gay" and "I am straight". Just a simple statement of fact devoid of shame or pride.  It's also plain that to expect this to be enough in the current world outside certain areas in the West is naiive.

The Twitter convo has some good points though, one in particular that Gay Pride, as an event, was a protest, but is now a celebration.  As a celebration, fair dos. As a protest, given my experience of shedding my homophobia, I think it would be ineffective now.  As another pointed out, the apparent need for so many to dress up as village people is kind of self-othering.  I think a much more effective protest would be an indication of sameness. A lot of gay people quietly marching and being ordinary would probably make the Cotswolds piss its collective pants.

I think proper homophobes are terrified of ordinary decent homosexuals, but are actually OK with obviously gay, camp, effeminate homosexuals.  The Larry Graysons, the John Inmans, the Maldwyn Pugh's.  There's safety there. A homophobe wants to know who's gay so he can, I assume, assume the backs-against-the-wall position in plenty of time.  But if their kind-of-mate, who they've been drinking with, or on a rugby trip with … if they turn out to be gay, then FUCK!… Anybody could be gay, how are they going to defend themselves from the legions of gays who might bum them?  Aaargh!

As an example, I know someone who thought Captain Jack in Doctor Who was quite good, and then saw Barrowman being all fabulous on a talk show, and their brain melted. How. Can. He. Be. Gay?

Let me get back on track...

The Achievement as a mutable state…

I think it's fine to be proud of something you've done. Something you've achieved.  The level of pride should be dependant on the achievement.  I'm proud of having fathered and part-raised two marvellous children. My wife has and continues to do more than half of of the raising part, you know because Patriarchy.  Let's face it though, the part-raising is a much greater achievement than the fathering. The fathering amounts to little more than occasionally squirting some semen into a vagina instead of a sock.

It's an achievement though. And I'm proud of it. At one point I was not a father, and then that state changed, and I was a father.

The mutable state of a homosexual…

The being a homosexual isn't, I think we've established, mutable.  If you're gay, you're gay and you were made that way.  What's mutable is the state of being homosexual being public. Being in or being out of the closet.

I think this is probably what's meant by gay pride. Not being proud of being gay, but being proud of the achievement of coming out of the closet. It must also be a tremendous relief.

If you're gay and you have hideous modern parents who are secretly pleased that they have a gay child, because it's one up on the McFarquhars, it's probably not a big deal. It's an achievement, but on the scale of my fatherhood.  Don't big up your pride too much.

There is no shame in being closeted in some circumstances, maybe even in most of the world. Being closeted is a legitimate defence against mental and physical pain. Against torture. Against death. An immutable state of being is a capital offence. That's fucking crazy.  If you're in some shitty place, because Nigel Farage's Mum isn't your Mum, then please, take pride in your closeted survival. And if you take a stand despite this, my word, you deserve a statue come the revolution!

  Tim Cook mentions being openly gay with friends and family. It's not a secret, but he's also not broadcasting the fact.  He probably felt no need.  But today he upped the stakes.  Today he broadcast the fact. To misquote Micky Flanagan, now he's out out. He did that because he concluded that this might help someone. This is an admirable thing. Something he has every right to be proud of.


Feel free to comment


 *I don't remember doing so, but Neil, if I ever called you a poof or a bender or laughed when someone else did, I'm deeply ashamed, I apologise. I sincerely hope it got better, whether you are gay or not. 

Mavericks, Win7, Parallels 10, shift key issue.

Since upgrading to Parallels 10, in advance of a planned upgrade to Yosemite*, I've been having trouble with modifier keys when in Remote desktop.
A few days in, and I'm annoyed enough to DuckDuckGo for a solution.

The answer was to set Virtual Machine -> Configure -> Options -> Advanced -> Optimize for games.

I found the answer here



*I'll wait until I have a 2 week break at Xmas, when I can do a full backup, install, test and revert if necessary, without risking an embarrassing Mac failure at work.

Python CodeEval boiler plate

I've been doing some CodeEval challenges. I've kept each in a small structured project. After a few, it was obvious that I needed to be able to run locally with my own test data and expected results, but be able to submit the source code unaltered. I have a folder for each challenge that contains three files. The source .py file, a small shell script and a file containing test data. This is what the boiler plate looks like ('balanced smileys' is the name of one of the challenges):

import sys

def balanced_smileys(message):
    """
    your code goes here, returning the result
    """
    
def code_eval(challenge):
    with open(sys.argv[1],'r') as filehandle:
        for line in filehandle:
            candidate_and_expected_result = [value.strip() for value in line.strip().split('| expected result =')]
            candidate = candidate_and_expected_result[0]
            expected_result = candidate_and_expected_result[1] if len(candidate_and_expected_result)>1 else None
            try:
                result = challenge(candidate)
                print(result)
                if expected_result:
                    assert str(result) == expected_result,'%s : Expected:%s, Got:%s'%(candidate,expected_result, result)
            except IndexError:
                pass 
       
code_eval(balanced_smileys)

The test data file (balanced_smileys.txt) looks like this:

:(:):)(::)(:()              | expected result = YES
:((                         | expected result = NO
i am sick today (:()        | expected result = YES
(:)                         | expected result = YES
hacker cup: started :):)    | expected result = YES
)(                          | expected result = NO

The launch script is simply this:

python balanced_smileys.py balanced_smileys.txt

When I make changes, I simply run:

sh balances_smileys.sh

This works well for most challenges and saves a bit of mucking about.

PEP 20 > PEP 8

While trying to become a better Pythoneer, I'm struggling a little with PEP 8. I'm isolated, so I kind of don't care, but at some point I may have to learn to play well with others, so I should. I got a reminder of PEP 8 during a brief attempt to use PyCharm. I've only very briefly used an IDE with any success. That was with Eclipse and Java. That was only with help from people and online tutorials. PyCharm pointed out the myriad ways I was breaking PEP 8. It didn't tell me the ways I was conforming to PEP 20. Some code from PEP 8:

class Rectangle(Blob):

    def __init__(self, width, height,
                 color='black', emphasis=None, highlight=0):
        if (width == 0 and height == 0 and
                color == 'red' and emphasis == 'strong' or
                highlight > 100):
            raise ValueError("sorry, you lose")
        if width == 0 and height == 0 and (color == 'red' or
                                           emphasis is None):
            raise ValueError("I don't think so -- values are %s, %s" %
                             (width, height))
        Blob.__init__(self, width, height,
                      color, emphasis, highlight)
 

And this is how I'd probably write it:

class Rectangle(Blob):

    def __init__(self,
                 width,
                 height,
                 color     = 'black',
                 emphasis  = None,
                 highlight = 0):
                 
        if  0 == width == height \
        and color == 'red'       \
        and emphasis == 'strong' \
        or  highlight > 100:
            raise ValueError("sorry, you lose")

        if  0 == width == height \
        and (color == 'red' or emphasis is None):
            raise ValueError("I don't think so -- values are %s, %s" %
                             (width, height))

        Blob.__init__(self      = self,
                      width     = width,
                      height    = height,
                      color     = color,
                      emphasis  = emphasis,
                      highlight = highlight)

From PEP 20, my intention is to hit the following:

Beautiful is better than ugly. Explicit is better than implicit. Readability counts.

And from PEP 8, I'm playing the trump card:

When in doubt, use your best judgment.

Some preferences, like extra spaces here and there, are obviously subjective. Objectively, my experience of writing and maintaining code in Ada and Pascal tells me to always be explicit about using named actual parameters. It's probably even more important in a dynamically typed language. If Python didn't allow me to do this, I'd likely have rejected the language. That is has this feature in common with Ada and the indented scope of Occam (No braces, Yay!) made it a no-brainer for me. The main problem would appear to be that it's longer. Less code fits on-screen at once. I've never found that to be a problem even when I was editing in eve on VMS from a VT220. This wouldn't bother me much, even though I think it's less readable:

        Blob.__init__(self=self,
                      width=width,
                      height=height,
                      color=color,
                      emphasis=emphasis,
                      highlight=highlight)

But this drives me insane:

        Blob.__init__(self,
                      height,
                      width,
                      color,
                      emphasis,
                      highlight)

If you don't understand why, the deliberate mistake should be enough to persuade you that in the majority of cases, you really want to be be explicit about those parameter names. I'd only an the exception for functions that only take one parameter. Look, this is still fine even though the order deviated from the formal parameter list:

        Blob.__init__(self      = self,
                      height    = height,
                      width     = width,
                      color     = color,
                      emphasis  = emphasis,
                      highlight = highlight)

And this is impossible to do sober unless it's deliberate. It's also easy to spot in a review or when debugging:

        Blob.__init__(self      = self,
                      width     = height,
                      height    = width,
                      color     = color,
                      emphasis  = emphasis,
                      highlight = highlight)

Idiotic Python

One of the problems of learning a programming language is that sometimes things one has written even relatively recently look idiotic.

Sometimes this is due to some bad advice on teh internets. Or perhaps misunderstanding an explanation.

I'd consider that I'm getting reasonably proficient in Python these days. I've had no formal teaching though, and I'm working in isolation, so I don't get feedback from colleagues or a mentor. I'm a lurker at the edge of the Python community.

I'm trying to improve.  I bought Writing Idiomatic Python by Jeff Knupp.  I read through it, skimming for things both do already and don't do/never knew about.  Things I already do were pat-on-the-back affirming, and I think confirmed I'm essentially on the right track.  The rest is a learning opportunity.

When there's a lull at work, which isn't often, I open the book, and pick something to check against in my current project. Yesterday I looked at Avoid comparing directly to True, False, or None and started searching for ==True; ==False; ==None; !=True; !=False; !=None; thinking there'd be a few to fix.  I found far more than I though would be there.  Most disappointingly, quite a few !=False and ==True.

I fixed a lot of this.  Sometimes it seems less readable, so not in all places yet.

A significant penny dropped though, even though I've read this section a few times and nodded to myself as though I understood.  It's the use of is not None to check whether optional parameters have been set.  Sadly, my code is riddled with well meaning but potentially harmful attempts to check optional parameters.

Another penny is teetering on the edge.  In trying to go from harmful to idiomatic, I've made a significant number of minor changes. Some of these alter the order of an if statement to remove negation from the condition:

if not weekend:
    work()
else:
    relax()

if weekend:
    relax()
else:
    work()

None of the changes are difficult, but there's risk of human error here which will be very difficult to find without adequate unit tests.  I don't have unit tests.  I've certainly tested as I've developed, but those tests were transient, not part of a test suite.

In a quick search on Python unit testing, I stumbled across a nice little intro from Jeff Knupp (again).

Sigh. I fear that retro-fitting unit tests is going to expose some unpleasant coupling.

Football is the World Game

I'm not an Association Football fan. I much prefer Rugby football.

I used to play for fun/exercise in my 20s. Not to any standard, but enough to be able to appreciate the game a little.  I watch Match of The Day sometimes, and usually get sucked in to The World Cup, because the standard is usually pretty high.

In the Internet age, this means that every four years, I get exposed to Americans belittling soccer.

http://daringfireball.net/thetalkshow/2014/06/16/ep-085

The real reason for this is indoctrination in a sport other than football.  It's simply cultural bias. It's why I prefer Rugby, why Texans prefer Gridiron, why Canadians prefer Hockey, or the Swiss prefer Tennis.

It's important to understand that football is the world game.  I suspect it's popular for two reasons. The rules are really pretty simple, so it can be grasped easily. The only equipment needed is a field, or a street or a car park; a football, volleyball, tennis ball, tin can, or even a stone. And, of course, jumpers for goalposts.  It's a game all almost all children, all over the world can play for free.  In unsupervised children's games, the only vaguely complex rule, offside, is discarded in favour of simply ridiculing goal-hangers.  Offside is enforced by peer pressure.

A common theme in disparaging football is that the one thing that separates us from animals is our hands, and hands aren't allowed.  I find this puzzling. The main thing that separates us from animals is our brains.  Other animals have hands, can use tools, are bipedal etc.  None can understand and play football.

I'd add something about baseball fielders needing catchers' mitts, which seems laughable to someone who's played cricket, but that would just be wallowing in the ignorance of my own cultural bias.

I switched from Markdown Pro to Macdown

The main reason: Unlike MarkDown Pro,  syntax highlighting is built in to MacDown .

MacDown is also free, although I'm perfectly happy to pay for good software I find useful.

The markup for code in MacDown is very simple:

This is a lazy stop-gap. I ought to be generating documentation from embedded docstrings automatically …but that's for another day.

Dynamically adding instance methods.

I've been playing with some code today. It works, but may well be too clever for its own good. It may be slightly insane. There may be much better ways to do this. I'd written a little class (SingleCommandSFTP) to make some other code a little shorter and neater. With one command, it was no problem, but when it expanded to three, the duplication of calls to __open and __close irked a little, so I added __run_command. The pragmatic thing to do was to stop. But unfortunately, I was having fun, and wondered about dynamically adding instance methods to call __run_command for all the available commands. I ended up with this:

class SingleCommandSFTP(object):
    """
    SingleCommandSFTP is a simple abstraction of the the Paramiko
    SFTP client.   

    http://www.lag.net/paramiko/docs/paramiko.SFTPClient-class.html
    
    e.g.
        sftp_client = SingleCommandSFTP(host     = "hostname",
                                        username = "username",
                                        password = "password")

        sftp_client.remove(path = '/mnt/a/b/file.txt')
    
    It is intended mainly for single operations as the
    connection is opened and closed for each command.
    For multiple commands it will be inefficient.
    """
    
    def __init__(self,
                 host,
                 username,
                 password,
                 port = 22):

        self.host     = host
        self.username = username
        self.password = password
        self.port     = port
        
        
    def __open(self):
        self.transport = paramiko.Transport((self.host,
                                             self.port))
        self.transport.connect(username = self.username,
                               password = self.password) 
        self.sftp_client = paramiko.SFTPClient.from_transport(self.transport)

        
    def __close(self):
        self.sftp_client.close()
        self.transport.close()
        
    def __run_command(self,
                      command,
                      **params):
        
        """
        Call to a paramiko.SFTPClient.'command' instance method.
        """

        log.info('sftp_client.{command}({parameters})'.format(command    = command,
                                                  parameters = params))
        self.__open()
        method = getattr(self.sftp_client,command)
        method(**params)
        self.__close()



    # instance methods using __run_command for all instance methods of SFTPClient
    # are added once, dynamically, below. 
    # SFTPClient Reference can be found at http://www.lag.net/paramiko/docs/paramiko.SFTPClient-class.html
    

def instance_method_code_string(object,attr):
    argspec = inspect.getargspec(getattr(object,attr))
    parameters = [arg for arg in argspec.args]
    if argspec.defaults:
        for index in range(len(argspec.defaults)):
           try:
               parameters[-1-index]+='="' + argspec.defaults[-1-index] + '"'
           except:
               parameters[-1-index]+='=%s' %argspec.defaults[-1-index]
    if argspec.varargs:
        parameters.append['*'+argspec.varargs]
    if argspec.keywords:
        parameters.append['*'+argspec.keywords]

    code_string = "def {name}({parameters}):\n".format(name       = attr,
                                                       parameters = ','.join(parameters))
    code_string +="    self._SingleCommandSFTP__run_command(command='{name}',{parameters}"\
                  .format(name       = attr,
                          parameters = ','.join('{p}={p}'.format(p=p) for p in argspec.args[1:]))

    if argspec.varargs:
        code_string +='*'+argspec.varargs
    if argspec.keywords:
        code_string +='**'+argspec.keywords

    code_string +=")\n"
    return code_string
    
def add_SFTPClient_equivalent_instance_methods_to_SingleCommandSFTP():
    
    instance_method_names = [attr for attr in dir(paramiko.SFTPClient) if getattr(paramiko.SFTPClient,attr).__class__==paramiko.SFTPClient.__init__.__class__ and attr[0]!='_']
    
    for instance_method_name in instance_method_names:
        code_string = instance_method_code_string(object=paramiko.SFTPClient, attr=instance_method_name)
        exec(code_string)
        setattr(SingleCommandSFTP,instance_method_name, eval(instance_method_name))


if 'instance_methods_added' not in dir():
    instance_methods_added = True       
    add_SFTPClient_equivalent_instance_methods_to_SingleCommandSFTP()
    

getattr, setattr, eval, exec, getargspec. I won't have bloody clue what this does in two weeks' time!

#RedirectToAppStoreAdsSuck

#RedirectToAppStoreAdsSuck

I'm getting a bit fed up with ads in my browser redirecting me to the App Store. So I decided to install the apps (if they are free), immediately remove them, and leave a negative review.

I suggest we all do this if we have a spare minute. The danger is that the apps climb the charts, but I found the process to be cathartic.

Feel free to cut and paste the text below into the review.

"I Installed and immediately deleted this app without opening it in order to leave this negative review to complain about being redirected to the App Store from an ad in a web page.

Seriously, this is an intensely intrusive, terrible way to try to get me to try your app.

Please stop!

#RedirectToAppStoreAdsSuck"