Musings of an older programmer

I’ve been lax the last six months on my blog, working instead on Campy (a C#/GPU programming language extension). Now that that is slightly under control, time to get back to the blog. And, by the way, the whole reason for Campy is to implement popular algorithms to run on a GPU, I thought I’d take some time to review what information is available on the internet on algorithms. The following is a list I’ve been working on for a few months. It is by no means a complete list. But, I hope it covers some of the more popular sites. The entries are not in any particular order. Note, this list does not include parallel algorithms, which will be a post unto itself, nor the seemingly required AI algorithms you must have nowadays.

In all resources, it’s important to evaluate the website for authority, search functionality, the range of implementation languages, and the number of algorithms listed. And a bit of skepticism, as I found over the years, many algorithms have typos and errors.

Compendiums

https://www.programmingalgorithms.com/

The home page of this website displays recently submitted implementations. However, at the top, the menu provides categorized implementations. The implementations provided are in a variety of programming languages, with examples. The code is not complete, and as is, won’t compile and run without driver code and framework. There are some glaring omissions. For example, binary and general tree algorithms are not at all included, and Tarjan’s SCC.

https://rosettacode.org/wiki/Rosetta_Code

This huge website has over 847 algorithms, implemented in hundreds of languages. A discussion of the algorithm is presented for each.

https://en.wikipedia.org/wiki/Rosetta_Code

https://en.wikibooks.org/wiki/Algorithms

This website is more like an introductory text on algorithm design and derivation instead of a website that enumerates algorithms. As such, it is difficult to find specific algorithms one may want to find.

https://www.toptal.com/developers/sorting-algorithms

A nice demonstration of different sorting algorithms with pseudo code.

http://bigocheatsheet.com/

A website describing the complexity of some basic data structures and sorting algorithms. Links to Wikipedia pages for the algorithms.

https://projecteuler.net/

As noted on the website, “Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve.” This site is a great reference for a class instructor looking for some standard problems to present.

https://dmoztools.net/Computers/Algorithms/

A directory of links to various websites on algorithms, mostly undergraduate and graduate class material.

https://xlinux.nist.gov/dads/

An excellent compendium of algorithms. Includes a Google search. Each algorithm description follows a standard format, and includes links to various implementations.

http://www.softpanorama.org/Algorithms/index.shtml

A website that is dedicated to help the motivated software developer in continuing education and dealing with various job-related problems. The list of algorithms is fairly nice, but it is directory oriented with no search.

http://www.geeksforgeeks.org/fundamentals-of-algorithms/

Similar to Softpanorama, and more with a more up-to-date feel, GeeksForGeeks has a good list of algorithms and implementations. The site has a number of self-study guides.

http://calgo.acm.org/

A list of algorithms collected over the years. Some of the algorithms are hard to follow, using archaic programming languages and pseudo-code.

http://www.netlib.org/

A collection of mathematical software, papers, and databases.

https://en.wikipedia.org/wiki/Algorithm

https://en.wikipedia.org/wiki/List_of_algorithms

What can be said about Wikipedia? A great resource for anyone.

 

Products

http://demonstrations.wolfram.com/index.html

As it says in its website, “the Wolfram Demonstrations Project is an open-code resource that uses dynamic computation to illuminate concepts in science, technology, mathematics, art, finance, and a remarkable range of other fields.” Although the implementations are in the Wolfram Language, they are an interesting read.

https://en.wikipedia.org/wiki/Wolfram_Research

http://www.algorithmic-solutions.com/

The LEDA system. Source code for free system available with purchase.

Textbook companions

http://www3.cs.stonybrook.edu/~skiena/combinatorica/

Companion website for Computational Graph Theory with Combinatorica.

http://jeffe.cs.illinois.edu/teaching/algorithms/

On online textbook of algorithms for a undergrad/grad class. No search.

http://algs4.cs.princeton.edu/home/

This is the companion website to Sedgewick’s book on algorithms (Amazon). It is a great website describing many important algorithms, although the implementations are not easy to read, and mostly in Java, which you have to sometimes fish around to find. But, there is a nice search text box tool in the main page, which can help you find specific algorithms.

http://www3.cs.stonybrook.edu/~algorith/

Companion website to The Algorithm Design Manual contains 70 of the most popular algorithms.

https://mitpress.mit.edu/books/introduction-algorithms

This book, known as CLRS, is a famous textbook used in countless courses. The book can be found online and employes pseudo-code to describe algorithms. It is an essential reference for any programmer or algorithm designer.

 

Reference aggregator and database websites

https://scholar.google.com/

https://dl.acm.org/

http://ieeexplore.ieee.org/Xplore/home.jsp

https://arxiv.org/

 

 

What the hell is Mono Mini?

Mono Mini is the LLVM JIT for Mono. It translates CIL into LLVM IR.

 

Why is it important?

Mono is a full implementation of C# that is platform independent, i.e., where programs can run on Windows, or Linux. The compiler for C# translates C# into CIL code, and Mono Mini translates CIL into machine code that can be executed. Mono Mini uses the LLVM backend, a project decades in development.

You may not care to know the details, but for my project Campy, which compiles C# code to parallelized GPU code, Mono Mini is highly relevant. Mono Mini has a simple parser of the CIL, and a translator to converted into LLVM. There are other libraries out there to compile CIL, like SharpLang and LLILC, but the source code for either are not easy readings.

 

Where is the code?

It’s located in the Mono Github repository. https://github.com/mono/mono

The source for Mini is in the directory mono/mini/. https://github.com/mono/mono/tree/master/mono/mini

 

What is Mono Mini written in?

Good ol’ primitive C!

 

What documentation is there?

http://www.mono-project.com/docs/advanced/runtime/docs/llvm-backend/

You will note very little information in regards to how to read the code, where to start. The section Architecture describes mostly what happens in LLVM, not the more interesting part of how CIL is converted into LLVM IR.

 

What LLVM API does Mono Mini use?

First, Mini uses a special fork of LLVM:  https://github.com/mono/llvm

Mini uses the LLVM-C API.

 

Where is the main instruction decode/translate loop to convert CIL into LLVM IR?

The main loop is in the function mono_method_to_ir(), located in mono/mini/method-to-ir.c. The loop decodes each instruction on the fly, switching on the opcode of the CIL instruction. The start of the while-loop occurs after quite a bit of stuff to set up a method. “while (ip < end) {” https://github.com/mono/mono/blob/master/mono/mini/method-to-ir.c#L7822

 

How is an instruction first decoded/parsed?

A large switch-statement in the main loop parses bytes for each instruction and creates an internal instruction. Most of the work is done in C macros that eventually call MONO_INST_NEW and MONO_ADD_INST.  For example, EMIT_NEW_ARGLOAD() is called:

#define EMIT_NEW_ARGLOAD(cfg,dest,num) do { NEW_ARGLOAD ((cfg), (dest), (num)); MONO_ADD_INS ((cfg)->cbb, (dest)); } while (0)

Expanding further:

#define NEW_ARGLOAD(cfg,dest,num) NEW_VARLOAD ((cfg), (dest), cfg->args [(num)], cfg->arg_types [(num)])

And further:

#define NEW_VARLOAD(cfg,dest,var,vartype) do { \
 MONO_INST_NEW ((cfg), (dest), OP_MOVE); \
 (dest)->opcode = mono_type_to_regmove ((cfg), (vartype)); \
 type_to_eval_stack_type ((cfg), (vartype), (dest)); \
 (dest)->klass = var->klass; \
 (dest)->sreg1 = var->dreg; \
 (dest)->dreg = alloc_dreg ((cfg), (MonoStackType)(dest)->type); \
 if ((dest)->opcode == OP_VMOVE) (dest)->klass = mono_class_from_mono_type ((vartype)); \
 } while (0)

And further, to define a structure to store the decoded instruction:

#define MONO_INST_NEW(cfg,dest,op) do { \
 (dest) = (MonoInst *)mono_mempool_alloc ((cfg)->mempool, sizeof (MonoInst)); \
 (dest)->inst_c0 = (dest)->inst_c1 = 0; \
 (dest)->next = (dest)->prev = NULL; \
 (dest)->opcode = (op); \
 (dest)->flags = 0; \
 (dest)->type = 0; \
 (dest)->dreg = -1; \
 MONO_INST_NULLIFY_SREGS ((dest)); \
 (dest)->cil_code = (cfg)->ip; \
 } while (0)

 

Right. Where is the actual CIL instruction converted into LLVM IR?

Well, first off, there is no CIL instruction. It’s actually represented in a peudo CIL instruction set (which I will call pCIL), computed as described above. For example, it contains things like OP_PHI and OP_MOVE, which do not exist in CIL.

The translation of pCIL to LLVM IR is in mini-llvm.c. The main loop for translation to LLVM IR is in process_bb(), https://github.com/mono/mono/blob/master/mono/mini/mini-llvm.c#L4143 , at “for (ins = bb->code; ins; ins = ins->next) {”  https://github.com/mono/mono/blob/master/mono/mini/mini-llvm.c#L4186 . In that for-loop, conversion is contingent on the op-code of the abstract CIL instruction, in the fat switch statement “switch (ins->opcode) {” https://github.com/mono/mono/blob/master/mono/mini/mini-llvm.c#L4275  .  Conversion is performed per method, in mono_llvm_emit_method(). “Emit LLVM IL from the mono IL, and compile it to native code using LLVM.”. That calls a sequence of functions, eventually to process_bb(), which contains that fat switch statement. Like this: mini_method_compile -> mono_llvm_emit_method -> mono_method_inner() -> process_bb().

Eventually, an LLVMValueRef corresponding to the end result of the instruction is created, and placed into stack-like variable, “values[]”. Each instruction works on the stack, as per usually translation to LLVM IR.

 

 

 

I’m going to tell a story that probably many of you can relate to.

Last week, I experienced a slow motion catastrophic failure in Windows. It surprise me because ever since Windows 7, I’ve had pretty good luck in rolling forward with the updates and upgrades of the OS. But, that changed when I moved to the Windows 10 Creator Update.

As Microsoft posted in its website, “Why wait? Download Creator’s Update now.” So I did.

Unfortunately, it started with problems with an old NVIDIA Ti 470 graphics card. More importantly, I couldn’t run a Samsung Android virtual machine with VirtualBox due to a problem in a VirtualBox network adapter driver.

I tried the usual: reinstalling VBox, some registry edits of some VBox settings, and sanity checks on the OS and disk drives. But, no matter what I tried, I couldn’t get VBox working.

Biting the bullet, I decided on increasingly drastic measures: a roll back to a previous restore point; a Windows 10 Reset (keeping user files). While the VBox problem was fixed, Windows broke in the process: I couldn’t use Windows 10 Start and type in “cmd” to run a shell.

In desperation, I decided to try a full Windows 10 Reset (removing all user files), but stopped immediately when Windows prompted whether I wanted to delete all data from all drivers in the machine! Really?? Being old school, I decided just to do a format/install fresh copy of the OS instead. I know, so “old school”.

In the end, I was able to get the PC back up and restore my files. But, I learned a few things about the tools MS offers to fix a Windows.

System File Integrity

SFC — System File Checker

Verifies the validity of Windows system files.

C:\Windows\system32>sfc /help

Microsoft (R) Windows (R) Resource Checker Version 6.0
Copyright (C) Microsoft Corporation. All rights reserved.

Scans the integrity of all protected system files and replaces incorrect versions with
correct Microsoft versions.

SFC [/SCANNOW] [/VERIFYONLY] [/SCANFILE=<file>] [/VERIFYFILE=<file>]
    [/OFFWINDIR=<offline windows directory> /OFFBOOTDIR=<offline boot directory>]

/SCANNOW        Scans integrity of all protected system files and repairs files with
                problems when possible.
/VERIFYONLY     Scans integrity of all protected system files. No repair operation is
                performed.
/SCANFILE       Scans integrity of the referenced file, repairs file if problems are
                identified. Specify full path <file>
/VERIFYFILE     Verifies the integrity of the file with full path <file>.  No repair
                operation is performed.
/OFFBOOTDIR     For offline repair specify the location of the offline boot directory
/OFFWINDIR      For offline repair specify the location of the offline windows directory

e.g.

        sfc /SCANNOW
        sfc /VERIFYFILE=c:\windows\system32\kernel32.dll
        sfc /SCANFILE=d:\windows\system32\kernel32.dll /OFFBOOTDIR=d:\ /OFFWINDIR=d:\windows
        sfc /VERIFYONLY

C:\Windows\system32>

Basic File System Integrity

Chkdsk — Checks a disk and displays a status report

This tool checks the validity of a file system. It used to check for bad sectors in a drive, but that is no longer performed by the tool.

C:\Windows\system32>chkdsk /?
Checks a disk and displays a status report.


CHKDSK [volume[[path]filename]]] [/F] [/V] [/R] [/X] [/I] [/C] [/L[:size]] [/B] [/scan] [/spotfix]


  volume              Specifies the drive letter (followed by a colon),
                      mount point, or volume name.
  filename            FAT/FAT32 only: Specifies the files to check for
                      fragmentation.
  /F                  Fixes errors on the disk.
  /V                  On FAT/FAT32: Displays the full path and name of every
                      file on the disk.
                      On NTFS: Displays cleanup messages if any.
  /R                  Locates bad sectors and recovers readable information
                      (implies /F, when /scan not specified).
  /L:size             NTFS only:  Changes the log file size to the specified
                      number of kilobytes.  If size is not specified, displays
                      current size.
  /X                  Forces the volume to dismount first if necessary.
                      All opened handles to the volume would then be invalid
                      (implies /F).
  /I                  NTFS only: Performs a less vigorous check of index
                      entries.
  /C                  NTFS only: Skips checking of cycles within the folder
                      structure.
  /B                  NTFS only: Re-evaluates bad clusters on the volume
                      (implies /R)
  /scan               NTFS only: Runs an online scan on the volume
  /forceofflinefix    NTFS only: (Must be used with "/scan")
                      Bypass all online repair; all defects found
                      are queued for offline repair (i.e. "chkdsk /spotfix").
  /perf               NTFS only: (Must be used with "/scan")
                      Uses more system resources to complete a scan as fast as
                      possible. This may have a negative performance impact on
                      other tasks running on the system.
  /spotfix            NTFS only: Runs spot fixing on the volume
  /sdcleanup          NTFS only: Garbage collect unneeded security descriptor
                      data (implies /F).
  /offlinescanandfix  Runs an offline scan and fix on the volume.
  /freeorphanedchains FAT/FAT32/exFAT only: Frees any orphaned cluster chains
                      instead of recovering their contents.
  /markclean          FAT/FAT32/exFAT only: Marks the volume clean if no
                      corruption was detected, even if /F was not specified.

The /I or /C switch reduces the amount of time required to run Chkdsk by
skipping certain checks of the volume.

C:\Windows\system32>

Bad Disk Drive Checks

There are a number of free tools which check for bad sectors on a hard disk drive (HDD). You should not use these tools on solid state drives (SSD) as these checks shorten the life of the SSD.

Many of the tools are manufacturer-specific programs. For example, Seatools is specific for Seagate drives. Sandisk SSD Dashboard is specific for Sandisk. Macrorit Disk Scanner seems pretty good tool with a colorful GUI to boot, and is not manufacturer specific.

Windows trouble shooter

Windows provides some trouble shooting programs to help detect and fix problems. You can operate them from the GUI, or the command line.

msdt.exe /id name-of-check

Available checks:
AeroDiagnostic
NetworkDiagnosticsDA
DeviceDiagnostic
HomeGroupDiagnostic
NetworkDiagnosticsInbound
NetworkDiagnosticsWeb
IEDiagnostic
IESecurityDiagnostic
NetworkDiagnosticsNetworkAdapter
PerformanceDiagnostic
AudioPlaybackDiagnostic
PowerDiagnostic
PrinterDiagnostic
PCWDiagnostic
AudioRecordingDiagnostic
SearchDiagnostic
NetworkDiagnosticsFileShare
MaintenanceDiagnostic
WindowsMediaPlayerDVDDiagnostic
WindowsMediaPlayerLibraryDiagnostic
WindowsMediaPlayerConfigurationDiagnostic
WindowsUpdateDiagnostic

 

DISM — Deployment Image Servicing and Management tool

DISM enumerates, installs, uninstalls, configures, and updates features
and packages in Windows images.

C:\Windows\system32>dism /help

Deployment Image Servicing and Management tool
Version: 10.0.15063.0


DISM.exe [dism_options] {Imaging_command} [<Imaging_arguments>]
DISM.exe {/Image:<path_to_offline_image> | /Online} [dism_options]
         {servicing_command} [<servicing_arguments>]

DESCRIPTION:

  DISM enumerates, installs, uninstalls, configures, and updates features
  and packages in Windows images. The commands that are available depend
  on the image being serviced and whether the image is offline or running.

GENERIC IMAGING COMMANDS:

  /Split-Image            - Splits an existing .wim or .ffu file into multiple
                            read-only split WIM/FFU files.
  /Apply-Image            - Applies an image.
  /Get-MountedImageInfo   - Displays information about mounted WIM and VHD
                            images.
  /Get-ImageInfo          - Displays information about images in a WIM or VHD
                            file.
  /Commit-Image           - Saves changes to a mounted WIM or VHD image.
  /Unmount-Image          - Unmounts a mounted WIM or VHD image.
  /Mount-Image            - Mounts an image from a WIM or VHD file.
  /Remount-Image          - Recovers an orphaned image mount directory.
  /Cleanup-Mountpoints    - Deletes resources associated with corrupted
                            mounted images.
WIM COMMANDS:

  /Apply-CustomDataImage  - Dehydrates files contained in the custom data image.
  /Capture-CustomImage    - Captures customizations into a delta WIM file on a
                            WIMBoot system. Captured directories include all
                            subfolders and data.
  /Get-WIMBootEntry       - Displays WIMBoot configuration entries for the
                            specified disk volume.
  /Update-WIMBootEntry    - Updates WIMBoot configuration entry for the
                            specified disk volume.
  /List-Image             - Displays a list of the files and folders in a
                            specified image.
  /Delete-Image           - Deletes the specified volume image from a WIM file
                            that has multiple volume images.
  /Export-Image           - Exports a copy of the specified image to another
                            file.
  /Append-Image           - Adds another image to a WIM file.
  /Capture-Image          - Captures an image of a drive into a new WIM file.
                            Captured directories include all subfolders and
                            data.
  /Get-MountedWimInfo     - Displays information about mounted WIM images.
  /Get-WimInfo            - Displays information about images in a WIM file.
  /Commit-Wim             - Saves changes to a mounted WIM image.
  /Unmount-Wim            - Unmounts a mounted WIM image.
  /Mount-Wim              - Mounts an image from a WIM file.
  /Remount-Wim            - Recovers an orphaned WIM mount directory.
  /Cleanup-Wim            - Deletes resources associated with mounted WIM
                            images that are corrupted.

IMAGE SPECIFICATIONS:

  /Online                 - Targets the running operating system.
  /Image                  - Specifies the path to the root directory of an
                            offline Windows image.

DISM OPTIONS:

  /English                - Displays command line output in English.
  /Format                 - Specifies the report output format.
  /WinDir                 - Specifies the path to the Windows directory.
  /SysDriveDir            - Specifies the path to the system-loader file named
                            BootMgr.
  /LogPath                - Specifies the logfile path.
  /LogLevel               - Specifies the output level shown in the log (1-4).
  /NoRestart              - Suppresses automatic reboots and reboot prompts.
  /Quiet                  - Suppresses all output except for error messages.
  /ScratchDir             - Specifies the path to a scratch directory.

For more information about these DISM options and their arguments, specify an
option immediately before /?.

  Examples:
    DISM.exe /Mount-Wim /?
    DISM.exe /ScratchDir /?
    DISM.exe /Image:C:\test\offline /?
    DISM.exe /Online /?



C:\Windows\system32>

Additional examples:

dism /online /cleanup-image /checkhealth
dism /online /cleanup-image /scanhealth

Windows Driver Verifier

Verifier.exe is a tool to check drivers installed on your system. It is a Desktop GUI application. Be very careful with this program: you can easily trash your system using the tool–from the voice of experience! Make sure to create a backup before proceeding. A good intro is here.

Recreate a Profile

Sometimes recreating a profile may solve your problems. See https://community.spiceworks.com/how_to/121165-re-create-user-profile-windows

How to Start Windows in Safe Mode with Command Prompt

Once in a while, you may need to go into “Safe Mode” when booting Windows. If the PC can boot, try this at the login screen: hold down the Shift key and click on the power button and then click on Restart. If the PC cannot boot, you must boot from a recovery disk. Make sure you do that before trouble hits! Plug in a flash drive, open Control Panel’s Recovery tool, then click Create a recovery drive.

 

Today, I’m writing this note about backup procedures for Windows. Up to now, I’ve been backing up my Windows machines (operating currently five) in an ad hoc manner. Cringe you may, but I’d only made a backup once a month or so. Well, for the many decades I’ve been working with computers, I’ve never lost that much, through disk failures, unexpected file deletions, and upgrades. If a machine got trashed, I could mount the driver in another machine, boot a good OS, copy the files I’m interested in, and start afresh with a reformatted/reinstalled OS. But, my luck changed unexpectedly last week.

The recent Windows 10 Creator’s Update (which is actually an upgrade) bit me big time. And, it took me a week to recover from the mess MS dealt me.

It seems that many updates from Microsoft these days are poorly tested. Updates to a box are shoved down your throat whether you like it or not. The “new” Microsoft is repeating the lessons it should have learned with Windows Vista.

Thus, backups are more important now than ever!

Backups in Windows 10

Microsoft says that Windows 10 provides a “backup” through “file history.” But this is NOT a backup. A backup should include system files and partitions, not just your personal files. Windows 10 also provides the “Windows 7 backup”, but I really do not understand why it’s called that, and not “Windows 10 backup.” Is it going to be supported in the future? Maybe not!

Online Backups

An online backup is a good alternative. There are plenty of commercial products available, some of which were reviewed just a few days ago (http://www.pcmag.com/article2/0,2817,2288745,00.asp).  However, online backups have several problems: the services require a yearly subscription; there are limits on the size or duration of the backups saved; transferring data to and from the online service places demands on the internet link; online backup providers may not be reliable in the long run.

Backup to a spare PC on a LAN

Because I have a spare box, several large disks installed in it, with the machine attached to a gigabit LAN, I went with Acronis True Image 2017 to the spare box. The cost of Acronis was $60, which isn’t too bad. On each machine, a script is run to wake up the spare server, run Acronis True Image, then shutdown the server at the end of the backup.

The script is executed by the Windows Task Scheduler, which has the ability to wake up the computer from sleep mode, or start the process when the computer is first turned on. This script is written in Powershell. I would have preferred to write it in Bash, but WSL Bash does not work with the Windows Task Scheduler.

function grep {
    $input | out-string -stream | select-string $args
}

$var = tasklist | grep TrueImageHostNotify.exe

if ($var)
{
    'There is a backup currently running.'
    'Cannot start another backup because it might interfere with existing backup.'
    Exit 1
}
else
{
    'ok'
}

'No backups currently underway.'
' Turning on server...'

.\wol\wol\bin\Debug\netcoreapp1.1\win10-x64\publish\wol.exe -mac 44-8a-5b-ca-6b-be -mount \\llano\e\ -hostname llano

if ($?)
{
    'Backup server is now up.'
}
else
{
    'Wol of the backup server failed for some reason.'
    Exit 1
}


# Invoke backup. Note, the script name is generated by Acronis True Image when you create a backup plan.

&"c:\Program Files (x86)\Acronis\TrueImageHome\TrueImageLauncher.exe" "/script:C8E6E46F-2075-4916-9AE5-04977FDBA37F"
if ($?)
{
}
else
{
    'Acronis True Image launcher failed for some reason.'
    Exit 1
}

# Wait until the backup to begin.
while ($true)
{
    $var = tasklist.exe | grep TrueImageHomeNotify.exe
    if ($var)
    {
        break
    }
    'Waiting for backup to begin'
    Start-Sleep -s 5
}


# Wait for backup to finish.
while ($true)
{
    $var = tasklist.exe | grep TrueImageHomeNotify.exe
    if ($var)
    {
    }
    else
    {
        break
    }
    'Waiting for backup to end'
    Start-Sleep -s 5
}


# Turn off the remote backup server.

'Turning off server...'
$uid = Get-Content .\uid.txt -Raw
$pw = Get-Content .\pw.txt -Raw
.\psshutdown.exe -d '\\llano' -u $uid -p $pw

Exit 0

A simple WOL program in Net Core

To wake up the backup spare box, I wrote a “WOL” program in C# Net Core. It sends out a UDP “magic packet” to wake up the backup server when needed, which must be configured in both the BIOS and the network controller driver to accept WOL packets.

namespace WOL
{
    using System.IO;
    using System;
    using System.Collections.Generic;
    using System.Net;
    using System.Net.Sockets;
    using System.Threading;
    using System.Linq;
    using System.Runtime.InteropServices;

    internal class Program
    {
        private static int _count = 1;
        private static int _interval = 1;
        private static byte[] _mac;
        private static bool _sleep = false;
        private static string _mount;
        private static string _hostname;

        public static void Main(string[] args)
        {
            try
            {
                ParseArgs(args);
                if (_mac != null)
                {
                    List<byte> result = new List<byte>();
                    result.AddRange(Enumerable.Repeat((byte) 0xff, 6));
                    result.AddRange(Enumerable.Repeat(_mac, 16).SelectMany(arr => arr));
                    byte[] buffer = result.ToArray();
                    IPEndPoint ep = new IPEndPoint(IPAddress.Broadcast, 9);
                    UdpClient client = new UdpClient();
                    client.EnableBroadcast = true;
                    while (_count > 0)
                    {
                        Console.WriteLine("Sending packet...");
                        client.SendAsync(buffer, buffer.Length, _hostname, 9);
                        if (_count > 1)
                        {
                            Thread.Sleep(_interval * 1000);
                        }
                        _count--;
                    }
                }

                if (_mount != null) Mount(_mount);

                if (_sleep) Sleeper.Suspend();
            }
            catch (MacException e)
            {
                Console.WriteLine(e.Message);
                Environment.Exit(1);
            }
            catch (UsageException)
            {
                Usage();
                Environment.Exit(1);
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                Environment.Exit(1);
            }
        }

        private static void Mount(string path)
        {
            var x = Directory.GetDirectories(path);
            System.Console.WriteLine(path + " size = " + x.Length);
        }

        private static void Usage()
        {
            Console.WriteLine($@"wol - Wake-On-Lan command-line utility
-c <int>              Number of packets to send to slave machine.
-i <int>              Number of seconds between packets sent.
-mac <string>         MAC address of the slave machine to wake up.
-mount <string>       UNC of the path of a slave file system to check if accessible.
-hostname <string>    Name of the slave computer on the LAN.
-s                    Indicates to place the host machine in sleep.
");
        }

        private static void ParseArgs(string[] args)
        {
            // Just in case, set up something.
            if (args.Length == 0)
            {
                throw new UsageException("No args. Check usage.");
            }
            int argc = 0;
            while (argc < args.Length)
            {
                if (args[argc] == "-mac")
                {
                    argc++;
                    string[] macParts = args[argc].Split(':', '-');
                    if (macParts.Length != 6)
                    {
                        throw new MacException(args[0]);
                    }

                    byte[] mac = new byte[6];
                    for (int i = 0; i < 6; i++)
                    {
                        mac[i] = Byte.Parse(macParts[i], System.Globalization.NumberStyles.HexNumber);
                    }

                    _mac = mac;
                    argc++;
                }
                else if (args[argc] == "-c")
                {
                    argc++;
                    _count = Int32.Parse(args[argc]);
                    argc++;
                }
                else if (args[argc] == "-i")
                {
                    argc++;
                    _interval = Int32.Parse(args[argc]);
                    argc++;
                }
                else if (args[argc] == "-s")
                {
                    argc++;
                    _sleep = true;
                }
                else if (args[argc] == "-mount")
                {
                    argc++;
                    _mount = args[argc];
                    argc++;
                }
                else if (args[argc] == "-hostname")
                {
                    argc++;
                    _hostname = args[argc];
                    argc++;
                }
                else
                    throw new UsageException("Unknown arg. Check command parameters.");
            }
        }
    }

    internal class MacException : ArgumentException
    {
        public MacException(string address)
            : base(address + " is not a valid MAC Address!")
        {
        }
    }
    internal class UsageException : ArgumentOutOfRangeException
    {
        public UsageException(string message)
            : base(message)
        {
        }
    }

    public class Sleeper
    {
        [DllImport("Powrprof.dll", SetLastError = true)]
        static extern bool SetSuspendState(bool hibernate, bool forceCritical, bool disableWakeEvent);

        public static void Suspend()
        {
            SetSuspendState(false, false, false);
        }
    }
}

 

Enjoy.

What is this Net Core everyone is talking about? Net Core is a cross-platform application framework for C#. With Visual Studio or the Net Core tool set, it’s not too hard to build server applications on any platform that can be deployed to any platform. This is remarkable considering the source code for C#/NET was proprietary code until Nov. 2014 when Net Core and parts of Net Framework were open sourced by Microsoft.

But, doesn’t Mono/Xamarin and Unity all offer a cross-platform application framework for C# as well?? While it’s true that Mono/Xamarin work cross platform, it was written from scratch. Mono was then extended to include bindings for Android, iOS, and other runtimes, and an API for UI common across selected platforms. Unity, as well, is a fork of Mono, with an API for graphics added. It too has differences with Net Framework that have emerged over time. It is expected that Net Standard, described below, will eventually supplant much of the functionality of these different cross-platform application frameworks.

Net Core has changed quite a bit in just the last few months let alone years. The file project.json, which specified the project dependencies, frameworks, targets, and runtimes, is now deprecated. Answers to questions about setting runtimes be found in Stackoverflow.com, but are confusing since they refer to the old project.json file.

First, a small side issue: WSL (Windows for Subsystem Linux)

If you are not interested in installing Visual Studio 2017 or Net Core on Windows, you can use WSL (Windows for Subsystem Linux) to easily develop a Linux from Windows 10.

After enabling developer mode for your Windows desktop, use “lxrun.exe /uninstall” and “lxrun.exe /install” to get a clean, fresh, up-to-date version of Ubuntu running on Windows. To run the shell, start up Powershell or Cmd, then type bash.

It’s also now possible to run X Windows applications in WSL with an X server installed on your Windows 10 desktop (https://sourceforge.net/projects/xming/). See Instructions1 and Instructions2. Some useful X Windows programs would be XTerm (sudo apt install xterm), gFTP (sudo apt install gftp)–just the tip of an iceberg of wonderful software.

Installing Net Core

For Ubuntu, see the instructions at https://www.microsoft.com/net/core#linuxubuntu to install Net Core in WSL.

You can install Visual Studio 2017 with the Net Core option, and develop Net Core applications from Visual Studio if you like.

Note, I’ve had problems getting the command line tool set working sometimes. This was because of multiple Net Core installations. Unfortunately, the newest Net Core package does not uninstall the older versions, at least on Windows. Beyond that, there shouldn’t be anything more to do to install. It all should find the dotnet executable, whichever platform you use.

Writing a Hello World program

To build a “Hello World” application, type the following in a WSL bash shell:

dotnet new console
dotnet restore
dotnet run

The first line creates a “Hello World” program. This may take a while while. The second command line performs essentially a “NuGet”, downloading and installing any additional packages. The third command line compiles and runs the program. Note, the product is a dotnet-dependent CIL/MSIL executable, and requires the “dotnet” executable to run, much like how Mono works.

To build a native OS executable for specific OS target, modify the CSPROJ file with a RuntimeIdentifiers tag.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
   <OutputType>Exe</OutputType>
   <TargetFramework>netcoreapp1.1</TargetFramework>
   <RuntimeIdentifiers>ubuntu.16.04-x64;ubuntu.14.04-x64;win10-x64;osx.10.11-x64</RuntimeIdentifiers>
  </PropertyGroup>
</Project>

Then, use dotnet restore/build with the -r option to build the programs.

dotnet restore -r ubuntu-14.04-x64
dotnet build -r ubuntu-14.04-x64
dotnet restore -r ubuntu-16.04-x64
dotnet build -r ubuntu-16.04-x64
etc.

In the directory bin/Debug/netcoreapp1.1/ubuntu.16.04-x64/ for example, you will find the executable dn. However, the executable is dependent on the dotnet framework being installed on the deployed machine.

To build a self-containing app with the framework contained with the executable, type:

dotnet publish -r ubuntu-14.04.x64
dotnet publish -r ubuntu-16.04.x64
etc.

In the directory bin/Debug/netcoreapp1.1/ubuntu.16.04-x64/publish you will find the executable dn with all the files needed to deploy the application.

Each time you change the target runtime list, use dotnet restore to install dependencies.

Relationship with Net Standard and other frameworks

Over the years, there have been many different implementations of the “NET Framework,” which resulted in different APIs. For example, in Mono/Xamarin/Unity, the Reflection API has several differences with the NET Framework Reflection API. Net Standard is a new base class layer that provides cross compatibility of different frameworks by providing a “shim” (or in my day, a “thunk”) that maps the functionality over the frameworks. The goal is to allow libraries written in one framework to be used in another framework.

Old:

New:

 

Compatibility

.NET Standard	1	1.1	1.2	1.3	1.4	1.5	1.6	2
.NET Core	1	1	1	1	1	1	1	2
.NET Framework (with tooling 1.0)	4.5	4.5	4.5.1	4.6	4.6.1	4.6.2	vNext	4.6.1
.NET Framework (with tooling 2.0)	4.5	4.5	4.5.1	4.6	4.6.1	4.6.1	4.6.1	4.6.1
Mono	4.6	4.6	4.6	4.6	4.6	4.6	4.6	vNext
Xamarin.iOS	10	10	10	10	10	10	10	vNext
Xamarin.Android	7	7	7	7	7	7	7	vNext
Universal Windows Platform	10	10	10	10	10	vNext	vNext	vNext
Windows	8	8	8.1
Windows Phone	8.1	8.1	8.1
Windows Phone Silverlight	8

Undoubtedly, you have probably seen the above figure showing the compatibility of the various NET Frameworks. But, this illustration simplifies what is actually required. To create a Net Standard library, you must compile for the appropriate Net Standard framework and provide a a list of dependencies for compatibility. See NetStandardLib and NetFrameworkApp in the example I provide.

I constructed a set of examples that check the cross compatibility of Net Core and Net Framework APIs with libraries in Net Core, Framework, and Standard. You can modify it to test out different scenarios.

Similarly, strict Net Standard 1.4 libraries can be called from Xamarin Forms libraries (Profile259), but no higher version number of Net Standard.

It is possible to compile your library to Net Framework, Xamarin, etc. while forcing your code to be Net Standard compliant by adding a <TargetFrameworks> tag to your .csproj file. For example, a <TargetFrameworks>netstandard1.4;net461</TargetFrameworks> tag will generate a Net Standard and Net Framework targets for your library. You could use a higher version number for Net Standard, but you would have to provide an additional target (<TargetFrameworks>) in the CSPROJ file to target the PCL (e.g., http://codinggorilla.domemtech.com/?p=1521), which may lead to other problems.

Examples can be found at https://github.com/kaby76/NetStandardCoreFramworkExamples

Additional Information

https://www.bartwolff.com/Blog/2017/03/20/targeting-multiple-net-platforms-in-a-single-nuget-package-w

http://stackoverflow.com/questions/43109292/reference-a-net-standard-1-6-library-from-a-net-4-6-2-library/43109941

http://www.ben-morris.com/sharing-libraries-between-net-core-and-net-framework-applications/

https://andrewlock.net/understanding-net-core-netstandard-and-asp-net-core/

gigi.nullneuron.net/gigilabs/multi-targeting-net-standard-class-libraries/

Virtually every website nowadays follows a God-awful trend in UI that generally replaces hyperlinked text with illustration-heavy art work arranged in a grid layout, making it not easier but HARDER for people to find a specific page. You cannot search for text on the page as it’s all now pictures; and, each page shows less links as a picture takes up more real estate than simple text.

Microsoft has joined that trend, making it almost impossible to find MSDN subscription downloads (https://msdn.microsoft.com/en-us/subscriptions/downloads). (Thank goodness Google search still works.) When you set your browser to http://msdn.microsoft.com, you get a large page that is nice to look at but shows virtually nothing:

Now, where are the downloads for your MSDN subscription? Click on the “Downloads” tab, and you get three items you can navigate to, none of which help.

All the other menu items in the page are the same–no MSDN downloads. if you click on the “Subscriber portal” in the upper-right, then you get a page with a 6 by 3 grid of icons for various downloads, but none of which say are an OS, or the VS C++ Build Tools:

From The Next Web, “In 2017, websites are likely to start moving back to basics and placing more emphasis on content. Whether that means we remove all of the other distractions we’ve spent years adding, or just making them take up less real estate is yet to be determined. Getting back to the heart of a website — the content — will be prevalent moving forward.” I certainly hope so. Maybe someone could define a measure that identifies when simple text is more appropriate than a grid of illustrations?

Please, let’s go back to the future and just use text where it’s appropriate!

If you’ve been working with Cygwin or MinGW, you may want to step over to Windows Subsystem for Linux (WSL) to take advantage of building and running tools in that environment. While the goal of Cygwin and MinGW has been to provide a Linux command-line tool set to Windows, it’s too easy to run the wrong tool (e.g., forgetting to install a tool in one environment, and picking up the identically named tool in another). In fact, many tools install their own private copy of MinGW (Git for Windows, SourceTree, Vagrant, …), so you find yourself constantly manipulating the search path.

Continue reading →

Honestly, is there any better way to spend a day (or two) on Pi Day (March 14 = “3.14”) than to write some code to compute the value of Pi?!

Continue reading →