The script has been tested on XP & 7 successfully.

For the linux side of things I run a python paster instance with the —reload flag that means paster will pickup changes and restart automatically, making deployment from my windows box a single click action.

Initially I started out a little rusty in my command line knowledge and had set about getting timestamps from first principles – lots of find, substring comparisons and messy embedded if control statements. Then I found command extensions which gave convenient access to file properties. Forfile and robocopy were potentially very useful shortcuts but neither are found in the base WinXP system so I eventually arrived at the following solution. The key part where datetime comparisons are made is in the main loop:

dir /b /tw /od !folder!%timestampfile% !file! | more +1 > %tmpfile%
set /p latest= < %tmpfile%
del %tmpfile%

Here the [tmp] timestamp file is compared with the current file in the recursive loop and ranked in ascending order of modification datetime (/b cuts out unwanted output from dir, /tw tells it to use file modification time for sorting and /od tells it to order by oldest first). This is piped to more which takes the bottom line (the newest file) and throws it into %tmpfile%. The variable !latest! then gets set to the value (/p switch tells set to expect a single line of input) and the temporary file is deleted. Afterwards we test to ensure !latest! is actually a file we want to transfer and not the timestamp file (or the batch file for that matter) and if so we copy it across to our transfer folder. Anyways:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

@echo OFF
 
REM IMPORTANT: 
REM 1. after you checkout your project into the working directory, create this batch file AND a blank txt file named %timestampfile% in your top level project directory. 
REM 1. make sure timestampfile IS unique (no other FILES with this NAME) within all subfolders of your working directory
REM 2. make sure tmpfile IS unique within pwd
REM 3. make sure tmp IS unique directory
REM 4. set your pscp, conn & pw strings TO appropriate values
REM
REM original post: http://aleatory.clientsideweb.net/2012/01/18/deploying-windows-dev-to-linux-server
 
set tmp=C:\tmp\
set tmpfile=tmp.txt
set timestampfile=timestamp.txt
set pscp="C:\path\to\pscp.exe"
set conn=user@server:/path/TO/directory
set pw=yourpasswdhere
 
REM ensure w2k command extensions are enabled AND delayed expansion FOR the control LOOP
setlocal enableextensions enabledelayedexpansion
 
REM tmp directory FOR copying modified FILES across
MKDIR %tmp%
CALL :strlen prelen %~dp0
set /a LEN=2+prelen
 
REM need TO copy timestampfile temporarily TO each subfolder FOR datetime comparison in main LOOP.
FOR /d /r %%j in (*) DO (
	copy %timestampfile% %%j
)
 
REM substring %var:~start,length% - 0-based
REM LOOP through all FILES in directory & SUB directories
REM IF last modified time greater than %timestampfile% - copy TO set TO ssh across
FOR /r %%i in (*) DO (
	set file=%%~dpnxi
	set file=!file:~%len%!
	set folder=%%~dpi
	set folder=!folder:~%len%!
 
	REM below will find last modified time FOR each AND RETURN latest file
	REM always returns 2nd file IF both were modified in the same minute, so we should PUT source file 2nd.
	REM t the moment /tw doesn't work as expected when comparing timestampfile with files inside different folders; 
	REM so the only immediate solution IS TO have a temp copy of timestampfile in the same directory ON each LOOP...
	dir /b /tw /od !folder!%timestampfile% !file! | more +1 > %tmpfile%
	set /p latest= < %tmpfile%
	del %tmpfile%
 
	REM IF latest isn't this batch file or the timestamp file then copy across, maintaining relative folder structure
	IF NOT !latest!==%~nx0 IF NOT !latest!==%timestampfile% ( 
		echo %tmp%!folder!!file!
		MKDIR %tmp%!folder! & copy !file! %tmp%!folder!
	)
)
 
REM timestampfile cleanup
FOR /d /r %%j in (*) DO (
	del %%j\%timestampfile%
)
 
%pscp% -batch -pw %pw% -r %tmp%* %conn%
RMDIR /s /q %tmp%
 
REM update timestamp file
echo %date%-%time%>%timestampfile%
 
GOTO :EOF
 
REM FUNCTION TO count 'string' length
:strlen <resultVar> <stringVar>
(   
	setlocal EnableDelayedExpansion
    set "s=!%~2!#"
    set "len=0"
    FOR %%P in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) DO (
        IF "!s:~%%P,1!" NEQ "" ( 
            set /a "len+=%%P"
            set "s=!s:~%%P!"
        )
    )
)
( 
	endlocal
    set "%~1=%len%"
    EXIT /b
)

(apologies for the non-batch file syntax highlighting – there’s a bug in GeSHi syntax highlighting engine that inserts HTML cruft into the code.)

You’ll note the intermediate step it takes of creating a temporary folder to place the files to be copied. This means I can copy the whole thing across as a oner rather than a long-winded new ssh connection for each one. The ideal windows solution would be to create a network folder using SAMBA on the Linux server where I could simply copy the files across in one go. If the file sizes were massive I might do this but I already use SSH for sysadmin so to cut down on one less potential security hole I’ll keep it simple.

Notably I had to use pukka REM statements and not the faster double colon to avoid the comments within the FOR loop being interpreted as commands and returning the following error:

The filename, directory name, or volume label syntax is incorrect

Also the one caveat with the above batch code is it won’t copy anything that has the same name as the script or the timestampfile value – so name them accordingly. If there is a better way of doing any of the steps I took then you are welcome to comment below.

Overall, yeah batch file processing is hard.

¹. pscp – a command line scp utility.

An insightful blog post¹ the other day railed against having lots of ideas and emphasised the need to stay on track, to concentrate on execution of The Big Idea. Here’s the other side of that argument.

There are many economic activities in which the rapid generation of ideas is a must have – take trading as one example. This is purely the game of projecting your ideas for where the market is going onto what is happening in reality. And just as you must see both sides of the trade as they say, so too do you need to foresee multiple scenarios ahead in order to understand all potential market movement. Because how it gets to the projected end result is just as important as when/if it gets there. Multiple & even contradictory ideas are the lifeblood then on which trading activity is based.

Sure, unlike startups it may be argued watching a single number make various shapes on computer screens is an inherently transitive activity. Execution is simply clicking a button. Perhaps, but the two are not binary opposites – far from it. What links the two, and thus supports the premise that lots of ideas can have a greater value than just one, is leverage. Traders piggy back on the economic activity of others just as startups can utilise the web services & APIs of others in order to create immense value with relatively little effort.

Take the likes of Zynga. Yes Zynga makes something. They churn out what has become mass media entertainment. But there’s plenty of developers doing that in Flash and still only getting a commission per title (if at all). Zynga on the other hand leveraged. They didn’t even create their first social game, YoVille, they just bought the thing inheriting it’s facebook userbase & social hooks and combined it with an offers service for monetization. This simple use of leverage provided the holy grail of a monetization channel for the social graph. It didn’t require enormously original or hi-tech games it just required leveraging services that were already available on the web. The idea – the route to market – was the important bit while the actual end-product was not i.e. the games are crap!

So just like the trader playing the trend, so too it’s worthwhile for startups to consider the prevalent mood and dare I say it go with the path of least resistance. Instead of seeking to single-mindedly disrupt it armed with The One Big Idea and seeking to create a platform from the outset, maybe being swift, nimble and able to jump on at the right time can also be the key to success. And for that you’re going to need a lot of ideas.

Refs:
1. “The Problem With Ideas”

I’m a longtime fan of netbooks over other mobile tech gadgets, finding their versatility an important stepchange in the prior arms race for increasingly powerful ‘portables’ that weigh a metric ton and need their own special carrying case, or mobiles with a preposterous array of unusable functions.

May have missed the hackathon at QUB but I’ve still managed to do a bit of hardware related fiddling with this guide on boosting my 4211′s performance & removing a niggle.

As refreshingly contrarian as a netbook can be, depending on one is a double edged sword – the more tasks you want to run on it, the greater the load on it’s relatively tiny engine. One of the few areas you can upgrade a netbook is it’s RAM. When messing around with portable versions of memory-intensive graphics apps, adding in an extra stick of DDR can boost performance significantly.

Similarly the more you depend on a gadget no matter where you go it will soon become encased in a fine layer of dust that destroys fan performance while general wear and tear of budget bits of plastic like the Acer 4211 will start to loosen moving parts and in the case of the fan it’ll be noticed by a pretty irritating clicking noise.

Tools I Used:
Small-head star screwdriver
mini knife
Leatherman Squirt P4
WD40

Removing dust & that fucking clicking noise from the fan

First take out the battery supply via the two slides on the underside of the laptop. Then unfasten the 7 screws dotted around the side edges of the bottom which allows you to unclip the black and silver halves, exposing the guts of the Advent. There were no seals broken in doing this so it’s doubtful as to whether this invalidates any warranty as is normally the case with fiddling with computer innards.

You can now see how big a mess has built up round the netbook fan and clean out accordingly. A toothbrush is recommended around the nooks and crannies while a cloth is fine for the rest of the surface area.

Now for that annoying noise the fan makes when it whirrs up after a few months of near continuous use. Simply spray some WD40 in around the fan mechanism itself, not to much is needed. Hopefully when the fan starts up again the oil will be spread around the fan parts and help prevent whatever friction may have been taking place.

Secondly the 4 fan screws binding the fan to the motherboard should be tightened to ensure this isn’t responsible for the noise as well. It turns out a couple of mine were pretty loose.

Adding more RAM

Upgrading RAM is likely the single biggest performance win on machines that have a free slot. Make sure when you purchase it’s of a similar tech as the existing bus and RAM – for example my Advent 4211 uses DDR2 which is incompatible with DDR and DDR3. Don’t rely on shop assistants to proactively check for you either, because in my experience they don’t. I got a PNY 1GB 533MHz module for about £20 but I seen a better deal on Amazon for a Crucial RAM stick at £15.

On the Advent it’s a simple task to fit it – locate the long black bus in the centre of the motherboard and the original OEM memory will be on one side with an empty space on the other. There’s even a cover sheet to protect the circuitry beneath your memory once you install it. Angle the stick so it lines up with the pins of the bus and fit it in ‘teeth’ first then snap the rest of the stick flat against the board, secured in place with the harness on either side.

Re-clip the base, replace the screws and battery then power back up to confirm the OS is running with the newly available capacity.

Sorted.

http://www.networkadvertising.org/managing/opt_out.asp

Incidentally I came by this information by way of Rapleaf, who are one of these ‘database marketing’ companies who engage in datamining browsing habits in a big way. Interestingly I remember them from a TechCrunch article a few years back where they started out innocently enough as an online social networking reputation tool – until eBay didn’t like it encroaching on their space and banned Rapleaf content from sellers auction pages.

If you’re still registered with them and like myself didn’t realise they had morphed into an marketing data company you can delete your account with them here.

Open up Adobe Reader 9 and click Accept. That’s it – you can close it and get back to using it in your browser from now on…

Perhaps, when attempting to convince users your browser is rock solid haxor-proof, it’d be advisable to demonstrate to them that your sites can detect which browser they’re currently running?

One workaround would be to save details to a db where they could be retrieved for analysis at a later time. This makes the data persistent but normally for debugging this is overkill. Log frameworks exist but sometimes incorporating third party software in is just bloat, especially in single developer projects.

The simplest alternative solution I can think of is to grab the request info needed and send it out to a mail address using PHP’s built in mail object:

$to = 'me@example.com';
$subj = 'Request variables';
$msg = 'Requested:\n';
foreach ( $_REQUEST as $k => $v )
 $msg .= "\t$k = $v\n";
mail( $to, $subj, $msg );

Mailing debug output to a free mail provider such as gmail means you have a persistent copy of the data (in chronological order) in a few lines of code without needing to start building up sql or think about clogging up your host with yet more logs. Is there a better way?

No ordinary decent virus this one though. It cleverly disabled my default browser – Chrome – coercing me into a specific set of steps that would ultimately place a rootkit on my OS. As my browser seemingly inexplicably was rendered useless, even after multiple uninstall/reinstalls, something else was up. Internet Explorer was attempting to connect to a “tolule.net” which on lookup resolved to a Chinese IP. So a quick entry into my Sygate advanced rules and I had a large swathe of Chinese IPs blocked. So I was safe for the time being giving me a chance to think about what was going on. (The Trojan was quite busy – attempting to connect every 10 mins or so and to multiple domains – initially always tolule.net but also gusmon.net and somemon.net – each time resolving to an address in China).

I ran a full AVG scan and two files were suspect – both resident in System Volume Information – the area where System Restore settings reside. Delete wouldn’t work (looking into it further if system restore is turned on these files cannot be manipulated) so I went for seemingly the next best option – system restore to a point before Chrome stopped working. Looking back this was a stupid thing to do but at the time seemed logical enough. A thought process that no doubt the hackers had already wargamed out in their heads prior to deployment. User sees app not working, reinstall doesn’t work, anti-virus identifies but can’t clear the Trojan, proceeds to system restore hence running the virus the hackers managed to plant somehow in Sys Vol Info.

Needless to say, the system restore failed, but now a plethora of hacked files had now successfully and relatively silently installed themselves on my OS. The rig included their own embedded db instance of Sql Anywhere, the a.exe worm that amongst other things engages in Process Hijacking described above, a bunch of reg keys that pointed device drivers and startup configs to the original hacked Macromedia directory, a hidden rootkit type process that ensured these keys and the directory were not tampered with and the actual downloader Trojan itself, W32/Agent.GBS!tr.dldr.

The process hijacking was something else. It would wait until I’d start an application and use the app’s virtual memory for it’s own devices. Hence why I was getting the Sygate reports of IE getting a life of it’s own. Wasn’t just browsers that were vulnerable, even MalwareBytes Anti-Malware software’s memory was hijacked. Each time the stolen app tried unsuccessfully to phone home to China.

By now it was obvious to me I had been infected with a particularly nasty virus and ran AVG a 2nd time. But now the Trojan was so well embedded AVG didn’t pick up anything more than a few measly HTTP cookies. I did wonder about Sygate at this stage but it still seemed to be successfully blocking all malicious packets so I was still just interested in removal. Spybot Search and Destroy was my next bet but it returned nothing. What’s more that TeaTimer process monitor it installs is a real resource hog – 128meg of virtual memory. That’s a bit more than a mere heartbeat in my book. Especially on my netbook! Got rid of that then was pointed to the MalwareBytes app by Matt @ BleepingComputer.com. The hackers obviously see it as a threat – after I installed it I tried to update it’s malware db but was prevented from doing so, presumably by the a.exe worm. It instead forced MBAM to attempt another connection to tolule.net. Proceeded with the scan anyway. Result? Every malicious file was identified.

Although still not out of the woods – MBAM would freeze when attempting to quarantine the audio driver reg keys whose values had been changed from wdmaud.drv to a dll file in the fake Macromedia directory. After I deleted a.exe and ran MBAM again, this time it succeeded in changing the reg keys after a restart. I had tried manual deletion of the reg keys and fake Macromedia directory prior to this but as soon as I closed regedit the keys and files would reappear. MBAM must have required the restart to remove whatever in-memory process had been running to protect the hacked data.

So that’s pretty much it. Quite an organised payload that, and a very well thought out attack vector relying on unwitting end user participation as much as the original web vulnerability.

Ok so really I’m just a petty net troll who completely overreacts to criticism online. That aside, I still cannot understand how the answering army at stackoverflow come to the collective conclusion that every question on a close-to-the-bone programming issue requires some inane form of rephrasing or just outright blanking.

I get over the former as SO is a lightning quick method of accessing really knowledgeable people on demand – but I had to endure a case of the later a while back when someone obviously thought the answer to the question I asked was too blase and proceded to provide a solution for a completely separate issue. And this despite me explaining exactly why I didn’t want his solution in the preamble. So I went and got a workable answer myself, posted it and accepted it as the solution. Job done.

Except this caused numbnuts to vote down my answer without explanation. So I voted his down, and told him why. Despite this, he questioned why I’d want to know what I wanted to know in the first place and voted down the question. It was at that moment that I realised SO, while largely self-moderating, is still missing the last 20%^TM required to remove the clinically insane from the process. It gives me no pleasure to disclose the most efficient solution currently is to multiply your web leverage in traditional fashion; create multiple accounts and hit back with a bewildering array of counter-comments and down votes…

It is embarrassing though when apparently throwaway questions asked on your secondary accounts are rated higher than your allegedly thought-provoking and succinct real persona *whistles*

In true Bileblog style, programmers appear to be sarky contemptible bastards who like nothing more than jumping on the inaccuracies of accepted thought; hence phrasing a question along the lines of ‘My colleague says x is no longer a good way to do things…’ will likely stir the hornet’s nest of pedantry as each contributor seeks to provide a more arcane answer as to why x sucks than the previous response. Recurse until someone mentions lambda.