Alleghany County took its name from the mountain range in which it is located. It was formed by act of Virginia Legislature on January 5, 1822 from parts of Botetourt, Bath, and Monroe counties. At that time Monroe was a Virginia county and remained so until the formation of West Virginia in May 1862 during the war between the states.

The County comprises approximately 452 square miles of which nearly 50% is in the National Forest. The County surrounds one city, Covington, which forms the principal center of population.

Covington, a city of the second class and the County seat, was named in honor of General Leonard Covington, hero of the war of 1812 and friend of James Madison and Thomas Jefferson. Covington was designated as a town in 1819 following a sale of lots from 25 acres of land owned by Dr. James Merry and in 1833 was incorporated as a city by the Commonwealth. Before the subdivision of Dr. Merry's land and naming of the town by Acts of Assembly, contemporary records refer to this site as "Mouth of the Dunlap".

The Town of Clifton Forge was originally called Williamson after the family which owned the land on which the town was located. However, in 1882 the Chesapeake and Ohio Railway Co. named its new depot there "Clifton Forge" and with the coming of the railroad it became not only a railroad junction, but a division point. After this, growth was very rapid and in 1906 received a charter as an independent city. In July 2001, Clifton Forge reverted from a city to a town.

The Town of Iron Gate was predicted to be one of the biggest cities in the Commonwealth, but this did not happen. Iron Gate grew from the development of the iron industry, and in the late 1880's a tannery was opened. The tannery operated until 1951.

Before the formation of Alleghany County, property records and court services for the area were provided from Fincastle, the county seat of Botetourt County. As this involved a two day trip for recordation of deeds or any court business, the formation of the Town of Covington a few years earlier made it desirable and perhaps necessary to provide a nearby location for these services. By further Act of the General Assembly, passed February 15, 1822, the County of Alleghany was attached to the Chancery District Court centered at Lewisburg.

One of the most influential persons in the formation of Covington and Alleghany County was Bernard Pitzer who owned and operated the first store in this area and bought from Uriah Humphries 217 acres lying at and west of the Mouth of the Dunlap Creek and Jackson River, now called Rosedale. Bernard Pitzer was assisted in his various activities by his large family. His brother, John Pitzer, Jr., was sheriff of Botetourt County, the most important local official at that time, during the period of the formation of Alleghany County and Covington.

From about 1790 to after 1825 the principal cash crop in this area was hemp. Hemp production was encouraged by the state with a bounty being paid from each unit delivered. Locally produced hemp was hauled by wagon to a rope factory in East Richmond. As ship stores accumulated, prices for hemp declined and agriculture shifted to grains, hay, and livestock. Early farmers of the area aimed to be as self-sufficient as possible.

According to the Census of 1840, the County's total population (including Covington and what is now Clifton Forge) was 2,749. Of this number 2,142 were "white free persons," 60 were "free colored persons," and 547 were slaves.

In the Civil War, Alleghany County furnished more soldiers to the confederacy than it had voters. The County suffered greatly in the war due to its location and many years were required for recovery from the losses sustained.

The biggest boost to industrial progress in the area was the decision in 1899 by the West Virginia Pulp and Paper Company to locate a mill at Covington. The coming of the pulp mill stimulated the development and growth of other industrial and commercial interests.

The government of Alleghany County has generally been very progressive in developing and supporting policies tending to economic growth and provision for improved services for the citizenry. An essential element for this is to encourage cooperation between agencies of the area where it is mutually advantageous to do so. At the present time, there are many cooperative activities between the jurisdictions.