Thanks to the anonymous commentor on my earlier post who called my attention to the correct sector size for the EADS drives. It is 512 bytes, not 4KB, so I have removed my gnop configuration.Irrespective of the drive, I have finally been able to confirm that my current motherboard's ICH7 implementation does not support AHCI so I am definitely limited to ATA100 performance. With the bad drive out and restoring with zfs send/receive, I did see all of my drives peak at 100 MB/s from time to time. I can now reliably get in the box transfer of in the 35 - 45 MB/s range. This is more inline with my expectations.
I'm still of the opinion that if you are going to use zfs on FreeBSD 8.1 that 8 GB of RAM is the minimum. While I now have a reasonable solution with 2 GB, it has taken an innordinate amount of my time to get it there. I hope others reading this will be able to learn from my experience and save a lot of time.
Without going into a lot of boring detail, the issue with the RAM is as much a problem with FreeBSD as it is with zfs. If you are doing large transfers (i.e. > 10X RAM), FreeBSD 8.1 will not release its Inactive memory fast enough and the cache will starve off thereby at a minimum impacting performance. The worst part of this is that FreeBSD will panic and shutdown from time to time. And since the FreeBSD support for my current board's ACPI isn't solid, the box locks up and doesn't recover without physical intervention. So from a stability point, it seems to me that you need to have enough RAM to make it unlikely that you will starve off the cache and put the OS in a position to panic. I am hoping, that this will be addressed in 9.0 but I'm not enough of a FreeBSD kernel geek to know.
Given that I am back to a sufficient level of performance and the reliability is good, as long as I don't try to remotely execute big transfers, I am going to hang with my current motherboard. In looking around a bit, I received some good recommendations regarding Intel's I3 as well as their current ATOM chips. I like the I3's; however, the cost is a bit more than I want to go at this time. The current 4 GB limitation on the ATOM keeps me from wanting to make an investment in them at this time. I'll keep my eyes open and when chip makers present a sufficiently interesting motherboard, I'll look at upgrading.
lbe
PS: if you are interested in what my current /boot/loader.conf looks like, here it is. Given the limitations of my current level of ignorance, I think this is about as good as it gets for FreeBSD 8.1 on a dual core Atom 330 configured as a NAS server using zfs.
kern.ipc.shmmax=67108864
kern.ipc.shmall=32768
vm.kmem_size_max="1024M"
vm.kmem_size="1024M"
vfs.zfs.arc_min="256M"
vfs.zfs.arc_max="784M"
vds.zfs.vdev.cache.bshift="16"
vfs.zfs.vdev.cache.zise="10m"
vfs.zfs.vdev.cache.max="16384"
vfs.zfs.vdev.min_pending="4"
vfs.zfs.vdev.max_pending="12"
vfs.zfs.vdev.aggregation_limit="131072"
vfs.zfs.vdev.ramp_rate="2"
vfs.zfs.vdev.time_shift="6"
kern.ipc.maxsockbuf=16777216
kern.ipc.nmbclusters=32768
kern.ipc.somaxconn=32768
kern.maxfiles=950000
net.inet.tcp.delayed_ack=0
net.inet.tcp.inflight.enable=0
net.inet.tcp.path_mtu_discovery=1
net.inet.tcp.recvbuf_auto=1
net.inet.tcp.recvbuf_inc=524288
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.recvspace=65536
net.inet.tcp.rfc1323=1
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.sendbuf_inc =524288
net.inet.tcp.sendspace=65536
net.inet.udp.maxdgram=57344
net.inet.udp.recvspace=65536
net.local.stream.recvspace=65536
net.local.stream.sendspace=65536
net.inet.tcp.sendbuf_max=16777216
aio_load="YES"
net.inet.tcp.mssdflt=9142
