Wednesday 30 July 2008

Mojave - the next "better" OS after Vista

If you think Vista is slow and love to moan about its shortcomings then you'll be glad to know Microsoft are gathering feedback on its 'next' OS, code named Mojave. Some users have been shown an early version - to hear what they think, headphones on and have a listen to some of the short video clips:

The Mojave Experiment

:-)

Mr C

Thursday 24 July 2008

TFS 2008/eScrum: adding new categories to a product backlog item

Further to this post on installing TFS 2008 and eScrum, here's how you add new "product backlog item" categories to the TFS process template (which can then be chosen for a "product backlog item" in eScrum).

Open up Team Explorer and choose Process Template Manager:

Choose the eScrum process template and select download. Choose a local folder and this will create a tree of folders containing all the template information. Go to eScrum\WorkItemTracking\TypeDefinitions\

Edit ProductBacklogItem.xml and modify the FIELD name="category" section. We added two new items.

<FIELD name="Category" refname="Microsoft.eScrum.Common.Category" type="String" reportable="dimension">
    <HELPTEXT>Category of the Product Backlog Item</HELPTEXT>
    <ALLOWEDVALUES expanditems="true">
        <LISTITEM value="Overhead" />
        <LISTITEM value="Document" />
        <LISTITEM value="External" />
        <LISTITEM value="Feature" />
        <LISTITEM value="Deployment" />
        <LISTITEM value="Defect" />
        <LISTITEM value="New to discuss" />

    </ALLOWEDVALUES>
</FIELD>

Save the file, select team explorer - process template manager again and upload the new template dir to the server (when prompted just choose your local escrum folder to upload from). Your new product backlog item categories will now appear in all new projects you create.

Clarkey

Thursday 10 July 2008

Installing Team Foundation Server 2008 and eScrum 1.0

I've been getting into Agile methods quite a bit recently, especially Scrum, which I am finding an excellent approach for managing projects and teams. We began initially using excel spreadsheets for the product and sprint backlogs, which worked quite well, but I'd heard about eScrum (process template for TFS) and, from what I'd read and seen at various seminars, looked to be a promising tool for providing a nice integrated tool for managing the various Scrum artifacts and processes.

Secondly, for some time I've been meaning to investigate TFS 2008 for team portal features, version control, item tracking and management of our builds.

We were about to start a new phase of a project so we discussed the pros/cons of piloting both TFS and eScrum, and decided this would be a good opportunity to try them both out. Were we biting off more than we could chew? Would it all be painful and a massive waste of resource?

This first post is all about the install and set up. Feedback on experience of using the tools can come in a later post ;-)

Installation of TFS

Top essential (and I mean essential) tip: read and follow the supplied TFSInstall.chm install guide file. This is key - if you are new to the TFS install and do not follow the supplied "checklists" for your chosen set up, you will endure pain, agony and spend many (mostly avoidable) dark hours in MSDN forums and KBs. We've all been there and it's not nice.

I pretty much tried to follow the install guide by the book, but below points out some specifics of our set up and the issues I came across.

Installation - What I did

Note, as this was a pilot, the install I chose was the (free) workgroup based Single Server TFS Install (i.e. TFS, SQL server, sharepoint and eScrum all installed on one box). TFS example project name in this case was called os-env, on a VM with hostname KSGV-TFS.

  1. Created a VMWare virtual machine hosted (based on a previous web windows 2003 server VM we have) on a VM hosting box. VM was called KSGV-TFS. Allocated 1.5 gig of RAM. C: system drive 20 gig. F@ drive data 60Gig.
  2. Disabled indexing service via computer management.
  3. Installed .Net 2 and 3 framework. IIS was already present on this VM image, tested and working. Uninstalled front page extensions (otherwise, as I found out (see in a bit), it halted the TFS install later).
  4. Ran windows update etc to bring the OS up to date.
  5. Installed FProt virus scanner v6 (old version had expired) and set to auto-update daily.
  6. Downloaded en_visual_studio_team_system_2008_team_foundation_server_workgroup_x86_x64wow_dvd_X14-29253.iso from MSDN subscription and extracted the files.
  7. IMPORTANT: then followed the instructions "Checklist: Single Server TFS Installation" EXACTLY as indicated in the TFSInstall.chm file in TFS install dir. Note I went for the single server installation (rather than separate front and back end servers) as for our 5 developer team on this pilot project, this seemed the logical and simplest route for the trial.
  8. Did not set up the admin user (TFSSETUP). As we were using local accounts and were using workgroup edition of TFS, the help recommends using your own user account (needs admin privs) instead of the TFSSETUP account as this uses up one of the valuable 5 free developers allowed in this free set up. So I set up and used local account dave.t.clarke (admin).
  9. TFSSERVICE, TFSREPORTS accounts were created and put in the Users group.
  10. Next followed the instructions to install SQL Server 2005 standard ed (again noted in TFSInstall.chm). I used a default SQL instance.
  11. Once installed, verified the SQL installation as indicated in the instructions, to check all necessary services were running.
  12. Installed SQL 2005 SP2. Then repeated step 10 (verify).
  13. Rebooted
  14. Installed TFS 2008 (I chose the option which would install Sharepoint too). Came across the infamous “front page extensions must not be installed” error during health check (despite having uninstalled it in step 2!). I love stuff like this... you know, you uninstall a component but it hasn't really gone. Great. A known error… in the KBs found out you can edit the metabase and remove IISFilters and IISFilter sections manually, which still contained refs to Front Page Ext. Uninstaller worked well there then... ugh.
  15. TFS health checks worked fine now
  16. When prompted entered the TFSSERVICE and TFSREPORTS accounts as applicable.
  17. Entered my SMTP email info. FYI, this is stored in C:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\Web Services\Services\web.config if you need to change it at a later date.
  18. Let it all install….
  19. Ran windows update. Picked up an update for sharepoint 3.
  20. Rebooted.
  21. Next I installed the pre reqs for eSCRUM (to be installed later in step 24):
    1. Microsoft Anti-Cross Site Scripting Library v 1.5
    2. Microsoft ASP.NET 2.0 AJAX Extensions 1.0
    3. Visual Studio 2005 Team Explorer (the eScrum 1.0 installer will check for v2005!). Had to download this separately as an img file and extract the files.
  22. Rebooted (otherwise I found the next TE 2008 install hanged in the next step).
  23. Installed Team Explorer 2008 from TFS 2008 install media.
  24. Opened Team Explorer 2008 to ensure it connected OK etc.
  25. Installed eScrum 1.0, by carefully following this ref: eScrum Install for TFS 2008 instructions . Although originally for beta 2, it seems to work fine on the final release of TFS 2008. I also gave local Users group MODIFY access to local C: windows/temp dir as per eScrum instructions.
  26. I used os-env for the eScrum project name in the above step, so I thus tested the WSS 3 eScrum site using the address: http://ksgv-tfs/Sites/os-env/default.aspx and tested the escrum web site using: http://ksgv-tfs:8080/escrum/.
  27. Both sites appeared to work fine at this point. Nice. Not as bad as I was expecting :-)

Setting up our Project in TFS and eScrum

I initially set the project up (see steps 24 and 25 above) using Team Explorer 2008 to create the project "os-env" in TFS and then added the entry manually (not ideal I know, but this is the way it works) into the eScrum RegisteredGroups.xml file, namely:

<?xml version="1.0"?>
<RegisteredGroups xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<isflatGroupCollectionDirty>false</isflatGroupCollectionDirty>
<Server Name="KSGV-TFS" Uri="http://ksgv-tfs:8080/">
<Group Name="os-env" />
<Group Name="another-project-here-etc" />
</Server >

</RegisteredGroups>

Update: you can also add projects to this file via the (rather obscure) http://ksgv-tfs:8080/escrum/admin.aspx page. Very basic interface with poor feedback and no validation, and make sure the name you enter is identical to the TFS project name you created using Team Explorer.

Once set up you will see the project in WSS 3 eScrum site using the address: http://ksgv-tfs/Sites/os-env/default.aspx and the escrum web site using: http://ksgv-tfs:8080/escrum/. See eScrum screenshot:

Setting up Basic Security

I then needed to set up the user security for our small pilot team. I opened up Team Explorer 2008 on the server, right clicked on our os-env project, selected Team Project Settings and added our relevant users to the groups Contributors and Project Administrators:

Both TFS and eScrum use these user lists for security (for example, on the eScrum web site you will not be able to add your team members to the "product team" list until these are set up).

I then needed to add the users to Sharepoint security (ie sharepoint site http://ksgv-tfs/site/os-env/, select site actions/site settings, people and groups) and also to SQL Reporting Services (http://ksgv-tfs/reports/, properties, security).

Check your security by testing the two sites (from another non-admin account machine) sharepoint eScrum site using the address: http://ksgv-tfs/Sites/os-env/default.aspx and the eScrum web site using: http://ksgv-tfs:8080/escrum/. With the latter, ensure you also try running the reports on the eScrum Reports tab to ensure SQL Reporting services security is configured correctly.

Setting up your development client

As we wanted to use our existing Visual Studio Team System 2008 Dev Ed to access/integrate with TFS for source control, work items etc, I had to install Team Explorer 2008 on my laptop (I initially, incorrectly, assumed it would be pre-installed as part of the VS 2008 TS). I installed it via the TFS install media but you can also get Team Explorer 2008 directly here.

Once installed, fire up Visual Studio, go to tools/options, select source control, and you will see Team Foundation Server is now in the list. Select it and VS will use this for your source control.

If you are worried about breaking older projects still in VSS (Visual Source Safe), you can easily switch back and forth between version control systems using this approach without any (noticeable at least!) ill effects. Do this before you open your relevant project though.

I then created a test project in VS 2008 and successfully added a test project's source to the TFS project os-env, just to ensure all was connecting OK.

You can then also select Team Explorer from within VS 2008 (View/Team Explorer):

I'll talk more about its usage in a future post.

Reducing the delays of eScrum report updating

Although all was now working fine, I notice that when we updated sprint items etc we did not see the changes for what seemed an age! Clearly some caching or scheduled warehousing was going on. Digging around I discovered that the eScrum reporting does not query the main TFS 'real-time' data but instead queries the TFSWarehouse database, which gets updated every hour (!). I thus changed this by going to the web service: http://localhost:8080/Warehouse/v1.0/warehousecontroller.asmx?op=ChangeSetting and updating the RunIntervalSeconds settingID to 600 (i.e. to update every 10 mins). Note you must run this from the local TFS machine.

Restart the "Visual Studio Team Foundation Server Task Scheduler" via Services in windows to pick up the changes.

You will then still find the SQL Server Reporting Services reports only update every 30 mins. To rectify this:

  • Went to SQL Server Reporting Services via: http://localhost/Reports/
  • selected the project os-env
  • selected properties tab
  • then selected each report and chose properties/execution. I then disabled the caching completely:

All done. At worst, now all reports will be 10 mins out of date. Depending on your needs and number of users, you may wish to still cache reports.

I then monitored machine cpu/memory usage for a while but did not see any ill effect of disabling the reporting cache and reducing the TFS Task Scheduler to 10 mins. I'll keep an eye on it though especially as the project grows in size. If you get any issues, maybe gradually increase the warehouse refresh time.

Bug Integration

In eScrum you can create links to bugs from a sprint task. See screenshot below:

In order for the "Query Bugs" button to work, you need to define the query that pulls out the relevant bugs from the database. See the eScrum help for more but basically you end up with a query similar to below that you copy/paste into the product bug setting section on the Product page (to use the query below in your own project just change the "os-env" project name reference to your own):

SELECT [System.Id], [System.AreaPath], [System.AssignedTo], [Microsoft.VSTS.Common.Priority], [System.Title], [System.State] FROM WorkItems WHERE [System.TeamProject] = 'os-env' AND [System.State] <> 'Deleted' AND [System.WorkItemType] = 'Bug' ORDER BY [System.State], [System.Id]

This will be run when you select "Query Bugs" in above screenshot.

Backups

With all our source code and product info going into TFS, I thought even for this pilot maybe backups would be a good idea ;-)

I took the approach of backing up the entire VM nightly to an external drive (and weekly to a network drive that is then backed up automatically by the standard corporate backup measures already in place). More on regularly backing up a VMWare VM in this post.

In Summary

Overall, the install all seemed to go fairly smoothly (especially considering all the pain I've read about on the net that others have gone through) and both TFS and eScrum appear to be working fine. I've created a Product, added team members etc, created a sprint and run some reports. Of course, the real test will be when we start to use this in anger over the next few weeks. The suspense is more than I can take...

It has to be said though, the installation experience is not the simplest - it is a manual process, with lots of jumping around in the help to see what to do next, manual config edits, lots of choices etc etc. However, if you are very careful and follow the supplied installation instructions, you will be rewarded :-)

Next post will be on usage and how we found both TFS and eScrum as day-to-day tools.

Summarised list of useful addresses

  • eScrum portal: http://ksgv-tfs:8080/escrum/
  • eScrum basic (very) admin page for adding a project: http://ksgv-tfs:8080/escrum/admin.aspx
  • Sharepoint project site: http://ksgv-tfs/sites/your-tfs-project-name/
  • SQL Server reporting Services: http://ksgv-tfs/reporting/
  • Web service to change the eScrum datawarehouse scheduler timing: http://localhost:8080/Warehouse/v1.0/warehousecontroller.asmx?op=ChangeSetting (note the TFS box localhost address)

And thanks to...

The above install was without a doubt aided by others sharing their valuable TFS install experiences (thank you guys, you saved me a considerable amount of pain):

Wednesday 9 July 2008

Backing up VMware Workstation VMs

We run a small number of VM instances which are mostly sandbox type environments and hence backups are not always essential, but if you are running something a bit more crucial and wish to back up a VM regularly, what are your options? Running 'traditional' third party backup s/w within the guest OS is one approach (and adopted widely) and more VM-centric enterprise solutions such as VMware's own VCB (VMWare Consolidated Backup) are also well suited but only available for VMware ESX.

Where does that leave us if we run VMware Workstation and want to back up the entire VM Guest 'image'? The easiest manual way of doing this is by selecting 'suspend' in VMware Workstation for the relevant guest, copy the VM files to another drive  (.vmx, .vmdk etc) and restart the VM. Easy. The con of this, of course, is that you are bringing down your VM for a short period but for most situations it does the job and has the advantage of being very easy to get the VM started again in a DR situation (i.e. simply take the backed up VM files and fire them up on a new host).

So how can we use the above 'suspend and resume' approach but automate it so it does this nightly for example? The command line comes to your rescue :-)

Before you start, ensure your relevant VM is not open on the desktop through the VMWare Workstation GUI, as this locks the instance.

Starting a VM instance from the command line

Use vmrun command with start parameter, for example:

"C:\Program Files\VMware\VMware Workstation\vmrun" start "F:\YourVMs\YourVMInstance\Windows Server 2003 Standard Edition.vmx"

Suspending a VM instance from the command line

Use vmrun command with suspend parameter, for example:

"C:\Program Files\VMware\VMware Workstation\vmrun" suspend "F:\YourVMs\YourVMInstance\Windows Server 2003 Standard Edition.vmx"

(for more switches, simply type vmrun with no parameters).

Creating a backup script

We've now got all we need to create a very simple windows cmd file, for example, create a VMBackup.cmd file and enter:

:: Suspend VM
"C:\Program Files\VMware\VMware Workstation\vmrun" suspend "F:\YourVMs\YourVMInstance\Windows Server 2003 Standard Edition.vmx"

:: ROBOCOPY (or xcopy) the files somewhere, pref a different box or network drive, for example
Robocopy.exe F:\YourVMs\YourVMInstance\ G:\externaldrive\backuparea\VMBackups\ /e /np /eta /r:1 /w:1 /log:F:\VMScheduledBackups\Logs\YourVMInstance-BackupLog.txt

:: restart the VM
"C:\Program Files\VMware\VMware Workstation\vmrun" start "F:\YourVMs\YourVMInstance\Windows Server 2003 Standard Edition.vmx"

You can now schedule this cmd file to run as a windows scheduled task in the normal way.  I use both a daily and weekly backup regime.

This is obviously a very simple backup approach and it does not alert backup failures etc (you've just got your trusty logs!) but for non-critical sandbox type environments it does the job nicely.

Clarkey