"Butcher shops" sell meat products, what is the name of a shop that sells cheese and ice cream?

The closest suggestion I can think of is creamery.

creamery - Technically a manufacturing plant that processes cream, but can be used to describe a retail outlet. Example.

Consider also:

dairy (New Zealand English) - A convenience store. Sells milk, ice cream, newspapers, cigarettes etc.

Can you go to the dairy to pick up some milk?

delicatessen (often shortened to deli) - A shop selling delicacies, including fine cheeses, and less commonly ice cream, and also meats, salads, olives, etc.

I've heard such a place called simply, "a dairy".

According to thefreedictionary.com, "dairy (n): 1. A commercial establishment for processing or selling milk and milk products."

If cheese is the predominant product, consider Cheesemonger :

A person who sells cheese, butter, and other dairy products.

This is chiefly BrE.

Caveat: Sometimes cheesemongers, in Britain and Ireland at least, use fancy French names like La Crèmerie or La Fromagerie.

Example usage from oxforddictionaries.com

From specialist cheesemongers to butchers, bakers, and fishmongers - we want to know about the smaller food retailers that provide the best produce.

A facility such as you describe used to grace the campus of the University of Vermont. (For the geographically distant, Vermont is a small dairy-producing state in the northeastern US. It is the home of US Senator Bernie Sanders.)

The store was called The UVM Dairy Bar. From the university website:

Generations of Catamounts have sweet memories of the UVM Dairy Bar, a hopping campus institution that dished out ice cream and other dairy provisions in Carrigan Hall for 45 years, from 1950 to 1995. Those who frequented the establishment as students will recognize a familiar face in Mary Dion, who served up countless cones and casual conversation at the Dairy Bar counter for over 20 years. Not long after the closure of its public outfit, Carrigan helped to aid the advance of another Vermont ice cream institution when Ben Cohen and Jerry Greenfield of Ben & Jerry’s made use of the test kitchen while they were in between facilities for a year.

(https://alumni.uvm.edu/foundation/thenandnow/3.php)

The term Dairy Shop seems uncommon, perhaps because ice cream (e.g., Ben and Jerry's) and cheeses (e.g., Cabot) are not commonly sold together in specialty emporia.

Nice idea, though...