Ellis, son of Edward Peppen Ellis and Susannah Mary Wheatley, was born in Croydon, Surrey (now part of Greater London). He had four sisters, none of whom married. His father was a sea captain, his mother the daughter of a sea captain, and many other relatives lived on or near the sea. When he was seven his father took him on one of his voyages, during which they called at Sydney, Callao and Antwerp. After his return, Ellis attended the French and German College near Wimbledon, and afterward attended a school in Mitcham.
In April 1875, Ellis sailed on his father's ship for Australia; soon after his arrival in Sydney, he obtained a position as a master at a private school. After the discovery of his lack of training, he was fired and became a tutor for a family living a few miles from Carcoar. He spent a year there and then obtained a position as a master at a grammar school in Grafton. The headmaster had died and Ellis carried on the school for that year, but was unsuccessful.
At the end of the year, he returned to Sydney and, after three months' training, was given charge of two government part-time elementary schools, one at Sparkes Creek, near Scone, New South Wales and the other at Junction Creek. He lived at the school house on Sparkes Creek for a year. He wrote in his autobiography, "In Australia, I gained health of body, I attained peace of soul, my life task was revealed to me, I was able to decide on a professional vocation, I became an artist in literature . . . these five points covered the whole activity of my life in the world. Some of them I should doubtless have reached without the aid of the Australian environment, scarcely all, and most of them I could never have achieved so completely if chance had not cast me into the solitude of the Liverpool Range.