At the outbreak of World War I in 1914, Gerard assumed the care of British interests in Germany, later visiting the camps where British prisoners were confined and doing much to alleviate their condition. His responsibilities were further increased by the fact that German interests in France, Great Britain, and Russia were placed in the care of the United States embassies in those countries, the United States embassy in Berlin thus becoming a sort of clearing house. From first-hand knowledge he was able to settle the question, much disputed among the Germans themselves, as to the official attitude of the German government toward the violation of Belgian neutrality.
At the request of Gottlieb von Jagow, after the fall of Liège, he served as intermediary for offering the Belgians peace and indemnity if they would grant passage of German troops through their country. On August 10, 1914, the Kaiser placed in Gerard's hands a telegram addressed personally to President Wilson declaring that Belgian neutrality “had to be violated by Germany on strategical grounds.” At the request of a high German official this telegram was not made public as the Kaiser had wished, but was sent privately to the President. After the sinking of the RMS Lusitania with many United States residents on board, on May 7, 1915, the United States ambassador's position became more difficult.

The German government asked him to leave the country in January 1917. Diplomatic relations were broken off on February 3, and he left Germany. He was detained for a time because of rumours that the German ambassador in America was being mistreated and German ships had been confiscated. When these rumors were disproved, he was allowed to depart. He retired from diplomatic service entirely in July 1917. He took up the practice of law in New York City. The George H. Doran Company of New York City published a book Gerard wrote on his experiences titled My Four Years in Germany released in 1917, My Four Years in Germany was turned into a film in 1918.