North Central Texas Wildflowers: a field guide

I am very delighted to announce that my mom, Mary Curry, has published a field guide to the wildflowers of north central Texas, featuring her superb photos for a huge number of wildflowers in the region!  She has taken beautiful photos that show key identifying features of each species, with both whole-plant views and close-ups of flowers, leaves, and stems (and multiple photos for almost every species).  The whole book is cross-referenced with the Flora of North-Central Texas, as well, for an easy route to additional information.  Her book is available on amazon; I highly recommend it!

"Winter is here"

So declared one of the Environment Canada weather bulletins last Thursday morning.  And several days before that, a hilarious "Winter is coming" preceded their more detailed forecast.  Sure enough, we had several centimeters/an inch or two of snow!

Snow, with dog for scale.  I first saw that phrase in an older paper about... sparrows?  grasslands?  I wish I could remember what paper it was, but it appears to be a not-uncommon unit of measurement, as a google search found several other people using this helpful scale.  19 November 2015, Winnipeg, Manitoba, Canada.

Error bars on points in R

Putting error bars on figures is surprisingly complicated sometimes (even in Excel...).  I have to google it every time for R.  Various packages can do it, but base R also has some functions that will help (segments and arrows).  I like arrows() because you don't have to draw extra bits to get the nice line on the end of your error bars, which it seems you need to do if you use segments().

#Plot your center points; you could use your averages or other value here.
#Play around with the values in x and y to see what numbers move which part.
plot(x=c(1,2),
     y=c(2,3),
     xlim=c(0,3),
     ylim=c(0,5),
     xlab="x",
     ylab="y",
     pch=21, bg="black")

#Here are error bars on the left point at (1,2).
arrows(x0=c(1,1),
       y0=c(2,2),
       x1=c(1,1),
       y1=c(1,3),
       length=0.25,
       angle=90)

#you can draw each error bar one at a time to get a better idea of how the code works.
#I show these in blue so you can see as it draws over the original bar on the plot.
arrows(x0=c(1),
       y0=c(2),
       x1=c(1),
       y1=c(1),
       length=0.25,
       angle=90,
       col="blue")

#top error bar.
arrows(x0=c(1),
       y0=c(2),
       x1=c(1),
       y1=c(3),
       length=0.25,
       angle=90,
       col="blue")

#Then let's re-do the plot and draw error bars for both points at the same time.
plot(x=c(1,2),
     y=c(2,3),
     xlim=c(0,3),
     ylim=c(0,5),
     xlab="x",
     ylab="y",
     pch=21, bg="black")

arrows(x0=c(1,1,2,2),
       y0=c(2,2,3,3),
       x1=c(1,1,2,2),
       y1=c(1,3,2,4),
       length=0.25,
       angle=90)

#You can see you end up with a list of points,
#so you can set these values from other calculations you make of
#standard deviation, standard error, confidence intervals,
#or whatever you are using.

Sorting, grouping, and selecting data in R

I got started sorting data in SQL.  Nice select functions where some variable equals some value and you can get distinct or unique values.  R confused me. I was delighted to find an R package that allows the use of SQL selects in R, but it can occasionally be a bit clumsy due to differences in table and object naming.  I kept seeing references to dplyr as the modern way to use R natively to organize my data.  So, I have decided it is time to start learning dplyr and already it is helping a ton.  I recommend starting with the dplyr vignette.  I also found this tutorial with different sample data to be helpful.  I tried out chaining (the mysterious %>% operator I have seen lurking in code occasionally) and it was fantastic.  No more weird intermediate variables!  The tutorial describes a different-package-specific version of chaining, but dplyr implements it as well (the help file says it was formerly '%.%' but the '%>%' version has become standard) so it worked fine even though I hadn't installed the other 'magrittr' package mentioned on that page.  So far the biggest help to me is the distinct() function, which gets unique combinations of factors as I get when I do simpler and simpler SQL selects (instead of getting repeated rows of categorical data when I try to subset using base R for some variable that does have additional unique variables that I am not currently interested in).

Installing libraries on a linux operating system

Now that my computer can dual-boot into Ubuntu and Windows, I am going to start getting things set up for my original need for a Linux-type operating system: a program for estimation of genomic clines called bgc.  Installing the GNU Scientific Library was first.  I searched for it in the software center, and I was surprised to have 15+ options in the search results.  I eventually chose "GNU Scientific Library (GSL) -- library package".  It noted in "more info" that "To compile your own programs you also need to install libgsl0-dev", which I figure I do since bgc requires compiling.  I installed that one too.  Easy enough!

HDF5 at first seemed like it would be similarly easy, but all the binaries ready for installing were for either Windows or CentOS Linux, which I assume would be different.  So I have to compile it.  I followed "Compiling the easy way" from the Ubuntu website.

Step 1.  I wasn't sure if I was supposed to remove the $ before $USER.  Turns out I was.  I got an error when I tried $claire for my user name.  "chown: missing operand after ‘/usr/local/src’"  It worked fine when I just put claire though.  The hdf5 instructions say I need 'gcc' to compile it.  This Ubuntu help says build-essential contains gcc, and the "Compiling the easy way" did have me install it in this step.

Step 2. Obtain the package!  I got a .tar with Unix line endings (since the other options were Mac or Windows line endings) and .tar.gz seemed more complicated.  I moved it to the /user/local/src folder (which is under "Home") and right-clicked, then "Extract here".  I looked in the folder and there seemed to be some configure files, so I'll go ahead and try the next step

Step 3. The link to auto-apt confused me, so i went with the manual version, installing apt-file and its update ("sudo apt-get install apt-file" then "sudo apt-file update" which did not take as long as the instructions threatened on my reasonably fast connection, maybe five minutes at most).  I went into my directory (/usr/local/src) in the terminal as instructed, and ran ./configure and it gave me a different error message.  I navigated one level lower (to /usr/local/src/hdf5-1.8.15-patch1) and then it worked!  It did not end with any dependency errors, but it also did not obviously with config.status: creating Makefile.  I scrolled back up and found it, though, along with lots of other creating of filenames in various directories, and "no obvious errors" as the Ubuntu page said, so on to Step 4.

Step 4. It says to give the command "make".  I did.  Many things began happening, including warnings about overflows and floating points and such.  However, it kept going, so I let it be.  The hdf5 instructions said to "make install" in their step 4.3.1 so hopefully that didn't mess it up.  It took maybe five minutes.  As I read the Ubuntu instructions further "sudo checkinstall" replaces "make install".  I did the sudo checkinstall and the terminal asked if i wanted to create a default set of package docs.  Only one option ("y") so I said yes.  Then it wanted me to write a description for the package and end it with an empty line or EOF.  I wrote hdf5 and then pressed enter.  But enter just gave me a new line prefixed with >>.  So, end of file (EOF) is ctrl+D in linux?  But a blank line might work too?  I tried the blank line first.  That worked (pressing enter twice to create the blank line).  Next up it gave me a list of values for building the package (maintainer, summary, name, version, etc).  I couldn't think of any changes so I pressed enter to continue.  Then I messed up and tried to copy the message and ctrl+c did something, not sure what.  So I ran "sudo checkinstall" again and it seemed happy to return to the same message.  I pressed "y" when it asked me this:

Some of the files created by the installation are inside the build
directory: /usr/local/src/hdf5-1.8.15-patch1

You probably don't want them to be included in the package,
especially if they are inside your home directory.
Do you want me to list them?  [n]: y

This got me to a weird screen with (END) at the bottom.  Help me, google!  Pressing Q and then enter got me back to the main screen where it asked:

Should I exclude them from the package? (Saying yes is a good idea)  [y]: y

The install failed.

dpkg-deb: error: parsing file '/var/tmp/tmp.d7tq23dvxW/package/DEBIAN/control' near line 7 package 'hdf5-1.8.15':
 error in 'Version' field string 'patch1-1': version number does not start with digit

So I tried again.

I keep forgetting and pressing ctrl+c.
*** SIGINT received ***

Restoring overwritten files from backup...OK

Cleaning up...OK

Bye.

According to wikipedia, if I understand correctly, this just interrupts the current process.  So, it should be okay.

Anyway, the new install with version number changed to 1.8.15-patch1 installed fine!

******************************************************************
 Done. The new package has been installed and saved to

 /usr/local/src/hdf5-1.8.15-patch1/hdf5-1.8.15_1.8.15-patch1-1_amd64.deb

 You can remove it from your system anytime using:

      dpkg -r hdf5-1.8.15

*******************************************************************

I'm still not really sure if that's where I wanted it installed, but this stackexchange answer seems to indicate it is an appropriate place and I think section 4.6 in the hdf5 instructions agrees.

Step 4.5 in the hdf5 instructions suggests testing the library, so I ran "make check" and waited for it to go through its paces, which took about three minutes.  It didn't return any errors so I appear to have installed it!   Soon I will compile bgc and see if that works; presumably if the hdf5 install didn't go right in non-obvious way, I will find out then.

They sound like geese

Greater Flamingo, Parc Ornithologique de Pont du Gau, Camargue delta, France, 01 August 2015.

Greater Flamingos in flight, Parc Ornithologique de Pont du Gau, Camargue delta, France, 30 July 2015. 

Castle surprise

Château de Tornac, southern France, 31 July 2015.

Current ruler of Château de Tornac (look carefully by the tower), 31 July 2015.

Life on the wall

Aigues-Mortes, southern France, 30 July 2015

Field season's over

Les Jardins de la Fontaine, Nîmes, southern France, 29 July 2015. 

Update to viewing and printing postscript files

A Windows 8.1 update to 'The Mysteries of Postscript'

Installing GSView and Ghostscript seems to work fine on Windows 8.1.  I haven't tested out the Inkscape connection because I never end up using it, as it seemed pretty inconsistent.  After installing Ghostscript and GSView with no problems, I set up the .eps printer as described in this tutorial.  However, at least with the driver I chose (first HP one in the list which was HP Color LaserJet 2500 PS Class Driver), there were NOT any PostScript Options under Document Options in the configuration step.  So I tried to go ahead and print my .eps file, and at least with the pdf I used, it worked fine.  I'm not sure if that has to do with Windows 8.1, or if I just happened to select a handy driver.

Sneaky plant

Needle-and-thread grass is starting to hitch rides with us on a regular basis as of early July.  02 July 2015, near Brooks, Alberta.

Moving default folders such as Pictures and Documents, etc. to the new partition

So I have set up my computer to dual boot into Ubuntu and Windows 8.1.  Then I partitioned my hard drive so I could have a shared data partition between Ubuntu and Windows.  Now, I want to change the default folders on both operating systems (like My Documents and My Pictures on Windows) to be on this new partition so I can use files from both places easily.  It was pretty easy and took about 15 minutes in Windows 8.1 despite my having delayed doing this for several days because surely it'd be hard. It appears to be essentially the same in Windows 7 if you have that.  I had three user accounts that I all directed to the same folders.  Otherwise it would have just been a few minutes.  I didn't redirect the desktop as I think this allows me to have different things saved on each one, and I only save temporary files on the desktop.  There was one conflict where I had the same file in each download folder, and Windows asked me what to do about it (replace, add, etc.) so I think if you try this after you have a lot of files it would still work.  I did this when my computer was essentially empty.


Okay, how to do this in Ubuntu?  It appears Windows always mounts the drive so it's visible, but in Ubuntu it doesn't and you have to configure it manually.  Here is a file showing all the commands and responses I got.

Next, how do I change the folders?  That first link I provided wanted me to make some sort of links, but this changing of a configuration file seems more permanent and unlikely to confuse programs like LibreOffice.  I couldn't find the  .config folder though and it was hidden.  Ctrl+H did the trick. I changed the file from $home/Downloads to media/Data/Downloads first to test it.  It didn't show up at first, but this older tutorial says I need to either log off and on again, or kill and restart Nautilus first.  My computer needed rebooting for some updates anyway, so I tried that first.  It worked!

I wondered if Ubuntu would politely accommodate the files from both the original default folder and the new folder like Windows did, so I put two documents in the default: one that had a duplicate and one that did not.  Then I went back to the configuration file and changed everything except the desktop.  Templates and Public did not have folders in Windows, so I added those to the data partition in case they are important for Ubuntu (I don't know yet).  This time I tried the killall nautilus in the terminal and it worked (much faster than rebooting or logging on and off).  Ubuntu did not move the files over, which makes sense since it wasn't particularly automated.  So if you already have a bunch of files, you'll need to move them manually.

I next noticed that there are two sets of links to documents, downloads, music, pictures, etc. in the files browser.  The top set (about the Data, Windows8_OS, and Computer partitions) go to the newly set folders.  The bottom set goes to the old /home ones.  I deleted the folders from /home, but then those lower links just gave an error for a missing directory.  I discovered you can right click on them and remove them.  No more confusion or errors.  I then tried saving something from LibreOffice to see where it goes as default.  It goes to /home/Claire which would have been annoying anyway I think.  I opened LibreOffice Writer and went to Options>LibreOffice>Paths and edited where "My Documents" is.  I tried installing another program afterwards (DigiKam for photo management) and it automatically went to my new Pictures folder.  Success!

I only have one Ubuntu user account right now so I just did this once.  I assume it'll work okay if I create another account, but so far I'm just using one account.

Sneaky sparrow

We've almost certainly heard this male Baird's Sparrow on several days since banding (same song in same territory), but he likes to perch in the grass out of sight.  08 July 2015, near Brooks, Alberta.

Setting up partitioning on my dualboot laptop

Now that I have my shiny new laptop successfully dual booting, I want to access the same files from both operating systems.  I see on the side menu thing (start menu?  task bar?) a "Windows8_OS" icon.  When I click this, it "mounts" the partition with Windows 8.1 so I can nose around in the files.  I had two screenshots from my dual boot adventures previously, so I went to those files and copied one and edited it.  Now I will reboot and go into Windows and see if it worked.  Yes!

I want a separate data partition though and inexplicably Windows 8.1 had 400+ GB in spite of using only about 35 GB currently.  I went to Adminstrative Tools>Disk Cleanup.  I selected all the files it gave me and told it to clean up system files as well (the button with the adminstrator shield on it).   Disk cleanup continued.  It took around 5-10 minutes and then brought up the window again with more selections for cleanup.  When I said okay, it asked if I wanted to permanently delete these files.  Yes, I do want to delete them.  Thanks.  It took at least 20 minutes.  Then I defragmented again.  This didn't get rid of the extra space.

Some googling let me know that Windows apparently stores some critical files at the middle/end of its original partition.  I followed these instructions to move the files into a smaller region.  It worked without a hitch.  I left about 70 GB for Windows 8.1.  Unfortunately, since I did my "unallocations" at two different times (once to make space for Ubuntu and now to solve the problem of why Windows wanted 400 GB), they are not combined.  The unallocated portions were separate.  It appeared the easiest way to combine them would be with the linux-based disk management GParted. 

I rebooted into Ubuntu and installed (using the application manager) and then opened up GParted.  It looks like I need to move the Ubuntu partitions around.  I did "swapoff" for the linux-swap partition, then moved it as described on the GParted site.  I just did one first to make sure it worked.  I rebooted twice into Windows and then once into Ubuntu to confirm everything was happy (the site suggests just doing Windows twice but I figured it was important to see if Ubuntu was happy since this was the linux swap partition I just moved).

Apparently I then needed a LiveCD version of GParted (can be done with USB too but CD seemed easier and it fit on the old CDs I have).  It was a bit over 200 MB so it took a while to download on my connection.  I burned this bootable CD on the lowest speed with "Verify disk after burning" checked.  I then reinserted the CD and shut down my laptop and turned it on again.  It didn't boot from the CD so I went to system settings, it took me through the Lenovo screen again, where I pressed F12 to choose the boot device.  I picked the third one, which had CD0 and DVDRAM in the name so I figured that was the CD.

I left it to sit a bit too long on default once GParted live had loaded and it started doing things.  Then the keymap screen came up as described. For the next few questions it asked me, I picked all defaults as suggested by the GParted help and then was at step 6.  I moved the remaining two linux partitions (/ and /home).  While I was there I right-clicked on the linux-swap file system and labeled it "Linuxswap".  After those changes were applied, I took the now-continuous and contiguous unallocated space and created a new partition which I labeled Data.

Once this was all done I rebooted into both operating systems to confirm I could see the files.  I ran into a bit of trouble with Windows not being able to read the Ubuntu screenshot, because it contained a colon.  I renamed the file back in Ubuntu and could see it fine in Windows then.

Finally, I needed to reset the changes I made from the initial instructions on reducing the size of the Windows partition.  The memory dump change wouldn't go without changing the paging file settings first.  It was still giving that error after I changed the paging file, so I did the other things (except for hibernation, which I have tried to turn off everywhere anyways).

I could have saved myself all this trouble by shrinking the Windows partition more appropriately BEFORE I installed Ubuntu.  Then I'd only have had to create the new partition from the one piece of unallocated space instead of moving things around.  Oh well, I learned how to use GParted!  Next up I'll figure out how to redirect all the default directories (My Documents, My Pictures, etc.) to the new partition and how to do the equivalent in Ubuntu.

Too early in the morning

Sunrise to the east.  Near Brooks, Alberta, 02 July 2015.

Moonset to the west.  Near Brooks, Alberta, 02 July 2015.

Another chapter from my dissertation out in the world: integrating hybrid zone models

I am pleased to announce that chapter 1 from my dissertation (highly modified and improved by reviewer comments) has been accepted as "An Integrated Framework for Hybrid Zone Models" in Evolutionary Biology.  It's been over five years since I first had this idea and it's finally out there.

Not a Savannah Sparrow

A white-tailed jackrabbit let me get quite close.   It was within the oil lease so perhaps it realized there was a fence between me and it.  Near Brooks, Alberta, 02 June 2015.

A younger pile of Savannah Sparrows

Four Savannah Sparrow nestlings.  Near Brooks, Alberta, 14 June 2015.

Switching gears from virtual machine Ubuntu over to dual booting

I still haven't gotten bgc to work, mainly because I haven't made it that far into Ubuntu/Xubuntu/linux.  Running the virtual machine on my old laptop was becoming aggravating because of how slow it was, so I decided to just wait until I got a new computer.  It's a Windows 8.1 Lenovo machine, so before I load it up with all my files and settings I'm going to attempt to get it to dual-boot the pre-installed Windows 8.1 and the very latest version of Ubuntu (15.04).  The current recommended version of Ubuntu is 14.04.2, but apparently that may sometimes conflict with Windows 8.1 still.  I downloaded 15.04, accordingly.

Back up Windows 8.1

Part of the reason I waited until getting a new laptop was that I didn't want to ruin my main working computer if the dual boot install fails.  I typed "Create recovery media" into the search and followed Window's instructions.  Part of this involved my having to go out of the house buy a 32 GB USB drive.  Sigh.

Creating the Ubuntu boot drive

To create a bootable USB (using only a 4 GB USB drive, by the way), I followed ubuntu's directions, including downloading the bootable USB drive creator program pendrivelinux.  I ran the program on my Windows 7 desktop and just followed the instructions.  I checked the format usb drive option.  Once I clicked create, a warning window with the list of changes popped up.

I'm not sure if 7-Zip is required for this (I already had it installed), but the pendrivelinux program brought up a 7-Zip window that began extracting the Ubuntu .iso file.  It's not mentioned anywhere so I assume it works without it, but if not 7-Zip is easy to download and install.

In the guide to installing Ubuntu with Windows 8.1 it mentioned that WUBI (Windows-based Ubuntu Installer) no longer works.  I had no idea what this was.  When my USB stick was done, it had a WUBI file on it, so I was concerned.  I think, from reading this and another article, that WUBI now has only limited functionality and formerly would only run if I try to run the bootable USB from Windows already started.  So it's still there, and assorted askubuntu posts exist about it, but it seems unlikely to jump out and get me.  Also, I left the USB stick in and left for several hours.  When I woke the computer up, Windows Explorer locked up.  I'm not sure if it was confused about the bootable USB, so I shut it down and removed the USB before turning it on again.

It turned out later in installation that my first bootable USB had two errors, so I remade it with UNetbootin.  Not sure if it was the program that did it or luck of the draw.

Partitioning the computer's hard drive

Alright, this is one of the parts that scared me.  First, Ubuntu suggests several procedures to prepare.  Right clicking on C:/ within "This PC" (the new "My Computer") doesn't get tools; "Tools" are a tab you can find once you right-click C and click "properties".  First I optimized/defragmented the drive.  It took maybe 20 minutes or so since it is brand new and I've only been messing with it for a few days.  Then there is error checking.  Mine came up with a message saying no errors have been found (I guess it checks often?) but that I could scan if I wanted.  Scanning only took about as long as it took me to write these few sentences.  It doesn't have the same options as shown on the Ubuntu site (for scan and repair), so I think their instructions are for an older version (most seem to be for Windows 7 and earlier, though they occasionally discuss Windows 8.

Okay, partitioning!  I thought the Windows Disk Management sounded easiest.  Their instructions are also a little out of date there.  I went to "Control Panel">"System and Security">"Administrative Tools">"Computer Management" (the instructions say "Disk Management", so it's a bit different).  In the left hand panel you can go to "Storage">"Disk Management".  It said I didn't have access to this on [COMPUTERNAME].  Hmm, maybe access from the admin account.  I logged out of my user account and into admin.  No protests this time.
Here's what it looked like.

So, now what?  I followed these steps and allowed 30 GB for my Ubuntu partition.  Twenty GB was recommended by Ubuntu but I added a bit more for installing other programs (suggested by the partitioning guide link).  Very quickly after pressing "Shrink", 29.30 GB became unallocated.  I restarted twice as one of those guides pointed out it needs to reconsider its space after changing the allocation.  I had a brief moment of panic as I tried to under my usual user password before realizing I was logging into admin and really, how would repartitioning lose my password while rebooting normally.  Anyway.  Then after reading more I found I should use Ubuntu to create the shared partition later, so I shrunk the Windows volume again (to the default allowed number) and rebooted twice.  It was unclear to me why I couldn't shrink the Windows volume beyond about 450 GB when only about 30 GB are currently being used.  The Disk Management told me to use the defrag application log.  Wasn't sure what to do there, so instead went to defragment again (under "Defragment and Optimize Drives").  That didn't change anything so that's a problem for a different day, if it is even a problem.  Edited to add: I should have shrunk down the partition at this step following this procedure, because it ended up adding another hour or three of work to move two unallocated partitions together.

Making sure hybrid shutdown/fast start is disabled

I noticed this when reading about partitioning.  Apparently Windows 8.1's automatic "shut down" is really a weird hibernation if "fast startup" is enabled (which is the default).  If the files are still in use by Windows 8.1's snapshot that is created, anything you mess with in Ubuntu will get corrupted and disappear.  It was fairly easy to turn this off, both at the power button settings and power plans.  Essentially this takes you through all the power options in Control Panel > Hardware and Sound > Power Options.  Make sure hibernating is never allowed under any of the advanced settings.  I found that it would hibernate if the battery was critically low as well, so I set it to shut down as well.

Installing Ubuntu

I think I'm finally ready now.  There are various guides that talk about problems with Secure Boot, UEFI, and other things I'm not really sure what they do.  However, this guide (which seems to do well on the askubuntu site) suggests that many issues are for older installs of Ubuntu.  This is a shiny new Lenovo as well, and it comes pre-installed with Ubuntu in some regions, so I decided I should just go ahead and try it now.  I shut down my machine and put in the bootable USB.  Here goes nothing!

The Lenovo splash screen comes up first.  To interrupt startup, press Enter.  I waited to see what would happen.  Oh.  Nothing.  Back to Windows login.  I restarted.  Nope.  Restarted again and pressed Enter at the prompt.  Uh.  Somehow that didn't work.  Probably should have pressed it longer.  Shut down again.  Hmm.

This makes me think I somehow still have fast start up enabled.  I press shift while shutting down.  Nope.  This is not where I expected to run into trouble.  I then tried F1, which is not what the Lenovo boot screen tells me to do.  I disabled Quick Boot under the Startup tab (the other option, which is now selected, is "Diagnostics").  I couldn't find an Intel Smart Response option.  So I shut down and started again.  Still doesn't boot from the USB.  I went to F12 during startup this time.  I selected the USB HDD option (it also listed the name of my USB, SanDisk Cruzer or whatever it's called).  I then got a GNU GRUB menu.

I selected check disk for defects first.  Now a prettier Ubuntu screen sits there checking a list of files.  Hmm.  Errors were found in two files.  I decided to remake the USB drive with the other recommended program.   No errors were found.  Reboot again, the selected "Install Ubuntu" from the GNU Grub menu.  (I recall a few sites saying it wouldn't boot UEFI, which is the new thing as opposed to Legacy/BIOS mode but mine seems to have turned out okay; I checked at the end).

Followed the assorted instructions, connected to wifi, and had it download updates while installing and install the third-party MP3 software.

The screen for "How to install Ubuntu..." alongside, erase Windows, or something else.  From googling, it was unclear to me if "Alongside" would work since I had already shrunk my Windows partition.  Selecting "Something else" brought up a very complicated table of what exists on my computer already.  Google time.  I followed these instructions (did not do the optional ones) to make three new partitions (swap, /, and /home) out of the unallocated large space (about 450 GB).  I left quite a bit of unallocated space because I didn't see an option to create a NTFS file system and the internet tells me gparted will let me do that once I have Ubuntu installed.  Once I press "Install Now" Ubuntu asks if I want to write these changes to disk.  Go back is default, but it looks okay so I say continue.

Things seem to go on.  It asks where I am, what keyboard to use, and for a username, computer name, and password.  I made sure the computer name was the same as what Windows calls it, but no idea if that matters.  Now it installs!  I'm doomed!  Time to go check on my bored helper.

Nobody ever does anything fun with her.

Optional while you wait: trim dog's toenails, sweep floor, take out trash, organize field work schedule for next day, cut and sample blueberry muffin cake.  I actually only made it through clip dog toenails, sweep part of the floor, and take out the trash before it was done and requested a restart.

It brought me to the GNU GRUB menu again.  I guess I still have it set to boot off the USB.  I just selected the top option, which was Ubuntu.  I logged in, looked around a bit, and shut down.  I removed the USB.  Now I get the Lenovo start screen.  It takes me to the GNU GRUB again.  Let's see if I can find Windows.  Windows Boot Manager is the third option.  I select it and get a scary error, which disappears quickly.  It can't find some file.  After a moment it goes back to the GNU GRUB.  I tell it to go to system settings and it goes through the Lenovo splash screen.  I press F12.  In the boot menu now, ubuntu is first followed by Windows Boot Manager and assorted other options.  PCI LAN is the selected option.  I select Windows Boot Manager.  The computer thinks... and it works!  I'm back in Windows 8.1.  I go have a look in Disk Management and things are wild now.

I shut down from Windows.  Upon startup, without pressing anything, it takes me to the BIOS.  I didn't do anything as I couldn't figure out what to do.  So I read some more, and then remembered a lot of sites said to disable secure boot.  So back to the BIOS.  Disabled secure boot.  Then the ubuntu GRUB menu was up again.  This time pressing Windows Boot Manager did not give an error!  One line said it was booting unsecurely, and then I am at the Windows login screen.  SCORE.  Oddly, it has changed the time and claims it is six hours later than it is.  I changed the time back.  I switched users to test all three out.  So far okay.  Shutting it down again.  "Booting in insecure mode".  I get the GRUB again, and click Ubuntu.  It runs some file checks.  No idea why.  I restart from Ubuntu instead of shutting down to see if that's how I can get the GRUB menu again.  Yup.  I waited too long though so it just automatically selects Ubuntu.  The screen does a really weird flicker when I am shutting down or restarting, shifting things from side to side rapidly.  This time on restart I pick Windows Boot Manager, and sure enough, there's the candy-colored stripes that now show on Windows.  I think I did it!  The time is correct in both operating systems now.

I also wanted to double-check that I did a UEFI install (and not the legacy BIOS) because both operating systems are supposed to have the same booting.  I clicked on the Files icon in the menu bar (I guess that's what it's called), then "Computer", and then navigated to sys/firmware/efi as described here.  The folder exists (with stuff in it, no less), so I seem to have done that correctly.

What now?
Good question.  Mostly, eat dinner.  This whole dual-booting procedure took me about four or five hours (troubleshooting included), but longer if you count several days of reading up on the procedures.  Next, I should probably read up on things one does in Ubuntu.  Software updates seem a good first step, although I think at least some should have installed during setup because I selected that option.  I just typed "update" into the search box and ran the first icon (a swirly A) which was Ubuntu's "Software Updater".  It found some updates for me.  Thanks, updater!  I looked at settings and it let me know that it would check for updates daily, including security fixes.  The second icon "Software and Updates" actually also took me to the settings menu from the Software Updater.

I guess the next things I should do are find out what type of user account I should have (I seem to be an administrator) and if they mean the same things as in Windows, create a shared partition for all my data and files, understand the security (anti-virus and firewall) a bit more, and then figure out how to install the scientific packages I need.

A lot of Savannah Sparrows lately

Two feathered Savannah Sparrow nestlings.  near Brooks, Alberta, 16 June 2015.

Savannah Sparrow closeup

This one was singing a weird song that from a distance sounded like a weird Baird's Sparrow.  Up close I could pick up the more typical buzzes at the beginning of the song.  It's got a dirty beak like most of the other Savannahs we've seen so far.  I guess they root around for food a bit.

Savannah Sparrow, near Brooks, Newell County, Alberta.  25 May 2015.

Color/colour band combination generator in R

Last time I did color band combinations I made a spreadsheet in Excel.  This time around I wrote a quick R script that generates the combos much more efficiently.  There is a SAS program available to generate color band combinations as well.  I've never tried it, but if you have more complex needs and SAS is available to you it looks great.  We just needed two colors on one leg so my R script does just as nicely for the simpler case.

#Color key:
#Red-R
#Orange-O
#Yellow-Y
#MediumGreen-G
#MediumBlue-B
#LightBlue-L
#White-W
#Gray-E
#Black-K

colours<-c("K","E","B","G",
           "O","R","W","Y", "L")

#load library (install it first if you don't already have it; I hadn't used 'gtools' before).
library(gtools)
#get all permutations
bandcombos<-permutations(n=9,r=2,v=colours,repeats.allowed=TRUE)
#n= number of colors, r=how many are adjacent, v= the vector of colors
#I allow repeats because I'm okay with two colors together (yellow over yellow, for example).

nrow(permutations(n=9,r=2,v=colours,repeats.allowed=TRUE))
#count how many combinations you've made.

setwd("C:\\Users\\YourUsername\\YourFileDirectory\\")
write.csv(bandcombos, file="yourproject_bandcombos.csv")
#Take your file and you can open it in a spreadsheet program or wherever for further formatting/use.
#We specified our bands would go on the left leg, so you could reuse all these combinations
#for the right leg.

Prairie crocus from May

It seems I mostly missed the prairie crocus blooming season this year.  They have really neat seed-heads though.  I found a big patch of them while out scouting sites in mid-May.

The wild bit is the seed head, and the purple flower in the right bottom corner is one of the few remaining blooms.  Northern Newell County, Alberta.  11 May 2015.

Savannah Sparrows are so easy to catch

They also tend to have a lot of dirt on their beaks.  Kinda messy critters.  Savannah Sparrow, near Brooks, Newell County, Alberta.  18 May 2015.

Chestnut-collared Longspur in the hand

Male Chestnut-collared Longspur near Brooks, Newell County, Alberta.  19 May 2015.

Field work in full swing!

Field work is in full swing.  I'm back out in Brooks, Alberta, and we're banding Baird's Sparrows, Savannah Sparrows, and Chestnut-collared Longspurs.  Here's one of our beautiful Baird's from yesterday!

Baird's Sparrow, near Brooks, Newell County, Alberta.  21 May 2015.

Beginning R: a very brief survey of methods for non-normal data

This was the last lab code for the quantitative methods class I co-taught with Dr. Nicola Koper this winter semester (i.e., the semester I used to know as "spring").

#Start by setting your workspace.  Even though we don't import a file this time,
#it's good practice so any figures you generate will go to the correct folder.
#Otherwise they tend to disappear into that totally obvious folder that you then forget.
######################################################################################
setwd("C:\\Users\\YourUsername\\")
#Your directory here.
#Or use the menu...
#Session>Set Working Directory>To Source File Location [or you can choose the directory]

#Today we'll use a sample dataset that's already in R,
#plus generate some random data of our own.
#No need for text files or importing!

#Generate some proportions
#No proportion data, so we'll just add it.
#I generate more data than we need, then cut off <0 and >1.
inventedproportions.prelim<-rnorm(mean=0.5,
                                  sd=0.5,
                                  n=300)
inventedproportions.keep.logical<-inventedproportions.prelim<=1&inventedproportions.prelim>=0
inventedproportions<-inventedproportions.prelim[inventedproportions.keep.logical]
airquality$inventedproportions<-inventedproportions[1:153]
#Generate zero-inflated data (download this package if necessary; I had never used it before).
library(gamlss.dist)

airquality$somanyzeros<-rZIP(n=153,
     mu=5,
     sigma=0.3)


str(airquality)
airquality<-na.omit(airquality)
str(airquality)
###########################################
#Brief review of graphical data exploration
###########################################

par(mfrow=c(2,3))
#Histograms for normality.
hist(airquality$Ozone)
hist(airquality$Solar.R)
hist(airquality$Wind)
hist(airquality$Temp)
hist(airquality$inventedproportions)
hist(airquality$somanyzeros)
#Residuals are what count for our models.
model.Solar.R<-lm(Ozone~Solar.R,
          data=airquality)
par(mfrow=c(2,2))
plot(model.Solar.R)
resid.model.Solar.R<-resid(model.Solar.R)
par(mfrow=c(1,1))
hist(resid.model.Solar.R)

model.somanyzeros<-lm(Ozone~somanyzeros,
                  data=airquality)
par(mfrow=c(2,2))
plot(model.somanyzeros)
resid.model.somanyzeros<-resid(model.somanyzeros)
hist(resid.model.somanyzeros)

###########################################
#Transforming your data
###########################################

airquality$ln.Solar.R<-log(airquality$Solar.R)
#add a constant if there are zeros. 
#pick a constant that transforms your variable best,
#see pp. 65-66 in Quinn and Keough for explanation.
airquality$ln.Solar.R.min<-log(airquality$Solar.R+min(airquality$Solar.R, na.rm=TRUE))
airquality$ln.Solar.R.001<-log(airquality$Solar.R+0.001)
airquality$ln.Solar.R.1<-log(airquality$Solar.R+1)
#Some other transformations.
airquality$Solar.R.inverted<-1/(airquality$Solar.R)
airquality$sqrt.Solar.R<-sqrt(airquality$Solar.R) #square root
airquality$cube.root.Solar.R<-airquality$Solar.R^(1/3) #cube root
airquality$fourth.root.Solar.R<-airquality$Solar.R^(1/4) #fourth root


#Box-Cox transformation.  R only lets you actually run this on a model.
#It's for transforming your response variable ONLY.
library(MASS)
par(mfrow=c(1,2))
Solar.R.boxcox<-boxcox(model.Solar.R)

#Use the graph to zoom in around the mostly likely lambda value.
Solar.R.boxcox.zoom<-boxcox(model.Solar.R,
                            lambda=seq(-0.05,0.35,0.1))
#From the plot, lambda should be about 0.15 (1/6 or 1/7, so a 6th or 7th root transformation.)
airquality$sixth.root.Ozone<-airquality$Ozone^(1/6)

airquality$arcsinsqrt.inventedproportions<-asin(sqrt(airquality$inventedproportions)) #arcsin square root transformation for proportions

#reflecting your data before transforming (pg. 66 in Quinn and Keough)
reflection.constant<-(max(airquality$somanyzeros)+1)
airquality$lnreflected.somanyzeros<-log((reflection.constant-airquality$somanyzeros)+0.001)
#Re-evaluating your data after transformation: run the exploratory analyses again.

par(mfrow=c(1,2))
hist(airquality$Ozone)
hist(airquality$sixth.root.Ozone)

par(mfrow=c(2,4))
hist(airquality$Solar.R)
hist(airquality$ln.Solar.R.min)
hist(airquality$ln.Solar.R.001)
hist(airquality$ln.Solar.R.1)
#Some other transformations.
hist(airquality$Solar.R.inverted)
hist(airquality$sqrt.Solar.R)
hist(airquality$cube.root.Solar.R)
hist(airquality$fourth.root.Solar.R)

par(mfrow=c(1,2))
hist(airquality$inventedproportions)
hist(airquality$arcsinsqrt.inventedproportions)

hist(airquality$somanyzeros)
hist(airquality$lnreflected.somanyzeros)

#Residuals are what count for our models.
model.sqrt.Solar.R<-lm(Ozone~sqrt.Solar.R,
                  data=airquality)
par(mfrow=c(2,2))
plot(model.sqrt.Solar.R)
resid.model.sqrt.Solar.R<-resid(model.sqrt.Solar.R)
par(mfrow=c(1,2))
hist(resid.model.sqrt.Solar.R)
#previous
hist(resid.model.Solar.R)

model.somanyzeros<-lm(Ozone~somanyzeros,
                      data=airquality)
par(mfrow=c(2,2))
plot(model.somanyzeros)
resid.model.somanyzeros<-resid(model.somanyzeros)


model.lnreflected.somanyzeros<-lm(Ozone~lnreflected.somanyzeros,
                      data=airquality)
par(mfrow=c(2,2))
plot(model.lnreflected.somanyzeros)
resid.model.lnreflected.somanyzeros<-resid(model.lnreflected.somanyzeros)

#compare
hist(resid.model.lnreflected.somanyzeros)
hist(resid.model.somanyzeros)
#The original is actually better... so remember to look at residuals!

###########################################
#Heteroscedasticity: inequality of variance
###########################################
#code from last lab, but with the new sample data.
pairs(~Ozone+Solar.R+Wind+Temp,
      data=airquality)

#density plots with our scatterplots
library(car)
scatterplotMatrix(~Ozone+Solar.R+Wind+Temp, #weird formula here, nothing as "response"/Y, and |indicates grouping variable.
                  data=airquality,
                  smoother=FALSE, #default is TRUE, draws a nonparametric smoother.
                  reg.line=FALSE, #default is TRUE, draws a regression line.
                  diagonal="density", #puts a density kernel line on the diagonal for each group.
                  # ellipse=TRUE,
                  # levels=c(0.5,0.95), #dataEllipse quartiles
                  pch=c(2,3,1), #rearrange the default points a bit
                  col=c(1,1,1)) #changed the colors to all black.

#tests for homogeneity of variance
#adapted from: http://www.cookbook-r.com/Statistical_analysis/Homogeneity_of_variance/

#These only work on groups, so group your Solar.R first.
airquality$Solar.R.cat<-NA
high<-airquality$Solar.R<=max(airquality$Solar.R, na.rm=TRUE)&airquality$Solar.R>=median(airquality$Solar.R, na.rm=TRUE)
airquality[high,"Solar.R.cat"]="high"

low<-airquality$Solar.R>=min(airquality$Solar.R, na.rm=TRUE)&airquality$Solar.R<median(airquality$Solar.R, na.rm=TRUE)
airquality[low,"Solar.R.cat"]="low"
airquality$Solar.R.cat<-as.factor(airquality$Solar.R.cat)

bartlett.test(Ozone~Solar.R.cat, #variable as a function of groups
              data=airquality )
?bartlett.test #gives options for other tests.

library(car)
leveneTest(Ozone~Solar.R.cat,
           data=airquality)
#Less sensitive than Bartlett's to non-normality.
###########################################
#Variations on regression
###########################################
#nonlinear relationships (adding a polynomial term), adapted from here.
#The I() function is necessary for R to read your quadratic or cubed term correctly.
quadratic.model<-lm(Ozone~Temp+I(Temp^2),
                    data=airquality)
summary(quadratic.model)

#cubic model
cubic.model<-lm(Ozone~Temp+I(Temp^2)+I(Temp^3),
                    data=airquality)
summary(cubic.model)

par(mfrow=c(1,1))
plot(airquality$Ozone~airquality$Temp)
ndQ<- data.frame("Temp"=seq(min(airquality$Temp),
                            max(airquality$Temp),
                            length.out=length(airquality$Temp)))

lines(ndQ$Temp,
      predict(quadratic.model,
              newdata=ndQ),
      lty="solid",
      col="black") 

lines(ndQ$Temp,
      predict(cubic.model,
              newdata=ndQ),
      lty="solid",
      col="red") 

anova(quadratic.model,cubic.model)

#weighted OLS regression
#Additional info.

#Group your data, or use existing groups.  (Here I just use the two groups from before for the airquality data.)

(var.solar<-tapply(airquality$Ozone, #the variable we're interested in
                      airquality$Solar.R.cat, #index (i.e., the grouping variable)
                      FUN=var, na.rm=TRUE))
solar.wts<-1/var.solar
airquality$wts<-NA
airquality[airquality$Solar.R.cat=="high","wts"]=solar.wts[1]
airquality[airquality$Solar.R.cat=="low","wts"]=solar.wts[2]

wt.lm<-lm(Ozone~Solar.R,
          weights=wts,
          data=airquality)
summary(wt.lm)
par(mfrow=c(2,2))
plot(wt.lm)
#compare with original
plot(model.Solar.R)
summary(model.Solar.R)

#Or run as ANOVA.
wt.anova<-lm(Ozone~Solar.R.cat,
             weights=wts,
             data=airquality)
summary(wt.anova)
plot(wt.anova)
#compare with unweighted.
solar.anova<-lm(Ozone~Solar.R.cat,
                data=airquality)
summary(solar.anova)
plot(solar.anova)

#For more details on how to do this (including normalizing your weights, which is apparently needed
#if you plan to get confidence intervals), check the top answer here.

###########################################
#Generalized linear models
###########################################
#functions, how to select family and link
glm.wolf<-glm(number~latitude,
              data=Depredations,
              family=poisson)
#uses default link for family.
summary(glm.wolf)
plot(glm.wolf)

par(mfrow=c(1,1))
plot(Depredations$number~Depredations$latitude)
ndW<- data.frame("latitude"=seq(min(Depredations$latitude),
                               max(Depredations$latitude),
                               length.out=length(Depredations$latitude)))
#plot the prediction with the new data (otherwise it uses rownumber and stretches the line out uselessly).
lines(ndW$latitude,
      predict.glm(glm.wolf,
                      newdata=ndW,
                  type="response"),
      lty="solid",lwd=2)

#GLM with a factor.
glm.snails<-glm(Deaths~Rel.Hum*Species,
                data=snails,
              family=poisson(link="log"))
#select the link manually, although still leaving it at default.
summary(glm.snails)

#quasipoisson suggested because very high deviance to df ratio.
glm.snails.q<-glm(Deaths~Rel.Hum*Species,
                data=snails,
                family=quasipoisson)
summary(glm.snails.q)

#plot the two lines.
plot(snails$Deaths[snails$Species=="A"]~jitter(snails$Rel.Hum[snails$Species=="A"]),
     pch="A",
     cex=0.5,
     xlim=c(60,80),
     ylim=c(0,20))
points(snails$Deaths[snails$Species=="B"]~jitter(snails$Rel.Hum[snails$Species=="B"]),
     pch="B",
     cex=0.5)

ndA<- data.frame("Rel.Hum"=seq(min(snails$Rel.Hum),
                            max(snails$Rel.Hum),
                            length.out=length(snails$Rel.Hum)),
                        "Species"="A")
lines(ndA$Rel.Hum,
      predict.glm(glm.snails.q,
                  newdata=ndA,
                  type="response"), #type="response" is necessary for these interesting links.
      lty="solid") 

ndB<- data.frame("Rel.Hum"=seq(min(snails$Rel.Hum),
                            max(snails$Rel.Hum),
                            length.out=length(snails$Rel.Hum)),
                 "Species"="B")
lines(ndB$Rel.Hum,
      exp(predict.glm(glm.snails.q,
                  newdata=ndB)),
      lty="dotted")
#Just to show that predict.glm is going the same thing as exp(prediction):
lines(ndB$Rel.Hum,
      predict.glm(glm.snails.q,
                  newdata=ndB,
                  type="response"),
      lty="solid",
      col="blue")

(predictions.compared<-data.frame(snails$Species,
                                 snails$Deaths,
                                 fitted(glm.snails.q),
                                 predict(glm.snails.q, type="response"),
                                 predict(glm.snails.q))) #note that without type="response" you get the linear predictors only.

#Run as an ANCOVA (analysis of covariance)
Anova(glm.snails.q, type=3)

#More info on Poisson regression specifically.

#evaluation of distribution fit using deviance (see lecture).
#Look at summary(yourmodel)
summary(glm.snails.q)
#Ratio of residual deviance to degrees of freedom as described in lecture.
#You can also look for packages that expand on this topic.

###########################################
#Nonparametric tests
###########################################
#Mann-Whitney/Wilcoxon tests.
wilcox.test(Ozone~Solar.R.cat,
            data=airquality)
#pairwise.wilcox.test() if you have paired measurements (i.e. repeated measurements), also known as the Signed-Ranks test.

#For two or MORE samples, use the Kruskal-Wallis rank sum test.
kruskal.test(Ozone~Month,
             data=airquality) #for two or more samples.
boxplot(Ozone~Month,
        data=airquality,
        xlab="Month",
        ylab="Ozone in some units")
#Opinions on post-hoc tests vary, but a few packages are available and easily findable via google.