anomalous territory

Isolate backups to a remote Linux/BSD/OSX server via “pull” over SSH

by admin on Aug.13, 2009, under backup, linux, software, ssh

My primary motivation for doing this was security. I’ve been reading some forensic logs lately and a great way to shoot yourself is if your critical servers perform “push” backups to elsewhere.

A typical example of this would be if I have a database server, and on this happy database server I have a crontab entry that creates a nightly snapshot of all the databases and then runs a little script to scp them all over to a backup server somewhere.

On first impressions, this sounds fine. You’ve got a production database server. You’ve got a remote backup server. You’ve got nightly jobs that run. Awesome! Well protected!

*bzzzzzz* You’re protected from innocent hardware failure, sure, but what happens when someone manages to compromise your database server and login? Regardless of their motivation, if they can read your nightly backup script, you are unbelievably screwed. They’re now not only pwning your production database server, but they have access to your remote backup server too and any historical data might as well be kissed good bye.

I wanted to be defensive and consider this scenario a “when” and not an “if”. My solution is nothing new, but it’s something I don’t see suggested much: Make the backup server PULL from the production database server using private/public ssh keys . For example: backup server has crontab entries that tell it to login to your production systems and copy the same databases you were previously telling the production servers to worry about. Also you can limit any commands the backup user’s login is allowed to issue on the production side. The backup server should also be behind a firewall, on a private network, isolated away from the outside world except for outbound connections in the direction of the production server.

Now not only do you have a happy database, you have an isolated backup server with absolutely no references to where or how the backups are stored.

An added benefit is that if you need to deploy any new production servers, all of their backup requirements can be centralized instead of adding multiple crontab entries to multiple new server at each deployment state. (It’s also a pain if your backup strategy changes greatly as you’d have to maintain many distributed references to your backup infrastructure.)

Obviously this is primarily for smaller scale systems. If you’re running Oracle 10g and have the infrastructure to support distributed RAID arrays full of redundant backups with offline tape libraries, well you’re probably here by accident then. However, if you’re running PostgreSQL or MySQL on a VPS somewhere, I hope this has been thought provoking at the least.

(Later I’ll post some actual How To code/commands to implement this.)

1 Comment more...

CakePHP 1.2 GROUP BY with HAVING COUNT(*) HABTM associations (limit the query based on associated data)

by volve on Mar.26, 2009, under articles, web

The problem: Your models look like this “Images HABTM Albums” so when you want to show 4 Albums, they come along with their Images too – great! BUT, what happens if some Albums don’t have any Images? You don’t want to show those to your users, but how do you filter them out? Well you could do it after the find() has run, in which case you may end-up with less than 4 useable Albums – bad :( To workaround that you could always ask find() to give you 8 Albums in the hope that at least 50% of them will be useable, but there’s no guarantee – still bad.

The solution below takes advantage of CakePHP 1.2’s bindModel() method to manually create an INNER JOIN in place of the pre-defined HABTM association. This allows us to execute the find() with a HAVING COUNT(*) in the group option. In the example below, this causes any Albums with 0 Images to be omitted from the results as they don’t meet the HAVING criteria. What’s more, the limit of 4 correlates exactly so 8-just-incase kludge needed.

This was driving me crazy so here’s the solution for anyone else that might be stuck in the same situation. (This method is in the Albums model.)

  function getAlbums($num=4) {
    $contain = array( // use Containable Behavior for efficiency      'AlbumsImage',
      'FilterImage',// although part of the magic and required, this seems to confuse CakePHP a little - I'll try and resolve it and update the post
      'Image',
      'Image'=>array('ImageSize',
                     'fields'=>array('Image.id','Image.title','Image.host','Image.path')),
      'User'=>array('fields'=>'User.id,User.username'));

    $this->bindModel(array('hasOne'=>array('AlbumsImage',
                                           'FilterImage'=>array( // FilterImage so as to distinguish between regular Image
                                                 'className'=>'Image',
                                                 'foreignKey'=>false,
                                                 'type'=>'INNER',
                                                 'conditions'=>array(
                                                     'FilterImage.status = 0',
                                                     'FilterImage.id = AlbumsImage.image_id')
                                                 )
                                           )
                           ));

    $albums = $this->find('all', array('contain' => $contain,
                                    'conditions' => array('Album.status'=>'0'),
                                        'fields' => array('Album.id','Album.user_id','Album.title'),
                                         'order' => 'Album.created DESC',
                                         'limit' => $num,
                                         'group' => array('Album.id','Album.title HAVING COUNT(*) >= 1'))); // here's the clincher

    return $albums;
  }
1 Comment more...

Linux/OSX mass file renaming

by volve on Jan.22, 2008, under linux

A handy command-line of text to mass rename files (in this case, change file extensions) that works in Linux and OSX:

ls -d *.cbz | sed 's/\(.*\).cbz$/mv -v "&" "\1.cbr"/' | sh

In this example, all .cbz files are renamed to .cbr (as I noticed some comic book archives I had were wrong).

An even more concise version using ‘basename’ was mentiond but OSX seems to have different default extension handling.

[via http://6v8.gamboni.org/Mass-renaming-with-linux-shell.html]

Leave a Comment more...

Extremely Useful Free Windows Software

by volve on Jul.07, 2007, under software

3 pieces of software that I’ve never really seen mentioned anywhere that I simply can not live without:

XnView

Free image browser/organizer with every feature of ACDSee and Adobe Bridge and many more. Just look at the operating systems it runs on: Windows, MacOS X, Linux x86, Linux ppc, FreeBSD x86, OpenBSD x86, NetBSD x86, Solaris sparc, Solaris x86, Irix mips, HP-UX, AIX – that’s dedication right there. I really do use this tool every single day, it’s simply that amazing. The Windows Shell extension alone is wonderful – instant thumbnail on the right-click menu of any supported file type. I don’t know why more people aren’t aware of this tool.
http://www.xnview.com/

CloneSpy

This brilliant tool lets you find (and automatically delete, if you wish) duplicate files in 2 or more locations on your system. You can do a full byte-for-byte compare, or simply rely on the filenames. You can also designate “pools” so only files listed in say Pool 2 will have the duplicate files deleted, should you wish to maintain any copies found in Pool 1 for example. A truly invaluable tool that’s saved me many, many hours of time and vast quantities of disk space!
http://www.clonespy.com/

RED (Remove Empty Directories)

This tool is fairly similar in function to CloneSpy above, except it works on empty directories and/or empty files. You specify a path for it to scan and it quickly goes through and presents a color-coordinated set of empty directories which it will happily remove with a confirmation click. It’s also smart enough to remove parent directories if after deleting empty sub-directories, said parent is also now entirely empty.
http://www.jonasjohn.de/lab/red.htm

Leave a Comment more...

New CNN.com Layout

by volve on Jun.30, 2007, under ui, web

I just noticed today that CNN.com have rolled out a new layout. I quite like the simplified look. It definitely feels as if it brings the news to the forfront whilst maintaining compartmentalization.

Looking at the brief stats it seems a lot lighter too. Oh and it has some smooth carousel action going on with the video thumbnails too.

Kind of a shame that Glasgow Airport terrorist crashing nonsense had to take me there but oh well.

Leave a Comment more...

Modern Keyboards = Not So Good

by volve on Dec.02, 2006, under ui

Funny article which I totally agree with: 10 Reasons Modern Keyboards are Shit

Keyboards really need to be updated. I know Dvorak tried but it was barely scratching the service. I’m sure there are enough wonderful touch-sensitive technologies out there that I shouldn’t have to move my hands around furriously to get some ideas down. Anyway, check out that article.

Leave a Comment more...

Cheeseburgers

by volve on Nov.29, 2006, under ui

Article worth a read: The Interface of a Cheeseburger

Leave a Comment more...

iTunes Media Wrangling with Smart Playlists

by volve on Nov.29, 2006, under media storage, software

UPDATE: It appears, at least in iTunes 7.0.2 that isntead of creating some big, cumbersome playlist with lots of “is not” entries as mentioned in the first link below, you can simple do “Playlist, is, Music” or “Playlist, is, TV Shows” – yay!

iTunes has some awesome features, but some odd deficiencies.

One such is the inability to build a Smart Playlist that contains ONLY music, or ONLY TV shows for example.

Although a handy solution is within your grasp. A useful feature of these Smart Playlists is to read OTHER playlists, so basically you create a big ol’ big list that excludes -for example- all TV, Movies, Podcasts, etc. then include that in your Music-only playlists going forward.

A handy guide explaining the aformentioned music-only base playlist:
http://www.43folders.com/2006/11/09/music-only-playlists/

Some other excellent iTunes wrangling tips of all varieties:
http://www.43folders.com/2006/11/10/smart-playlists-for-packrats/

Good luck and god speed!

Leave a Comment more...

scrybe organiser

by volve on Oct.22, 2006, under software, ui, web

This looks fairly impressive. At times it appears as though they’re using Flash or Flex, but at others it’s hard to tell. In any case, it should be fun to see. I’m particularly impressed with the couple of different printing options mentioned, and the smooth integration of timezones. I’m not so enamored with the Offline support as I think it’ll have many limitations, unless they’re definitely using Flex… hmmm

Teasing details and a beta email signup available at: Scrybe site

Leave a Comment more...

cellphone concept

by volve on Oct.08, 2006, under ui

I do believe this cellphone concept with full surface touchscreen looks /awesome/:


[via engadget mobile]

1 Comment more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!