She hadde passed many a straunge strem;
At Rome she hadde been, and at Boloigne,
In Galice at Seint-Jame, and at Coloigne.
She koude muchel of wandrynge by the weye.
 (General Prologue, I.464-67).

Pilgrimages began as exercises in penance, as defined in The Parson's Tale:

Commune penaunce is that preestes enjoynen men communly in certeyn caas, as for to goon peraventure naked in pilgrimages, or barefoot. (ParsT (X.105))

Going on a pilgrimage could be a penitential act. Walking barefoot or even riding a horse could be a difficult undertaking, along poorly maintained and dangerous roads. Journeys overseas, to Compostella or Jerusalem, were complicated, difficult, and dangerous. In the later Middle Ages, conditions of travel improved, but getting from England to Jerusalem (as did the Wife of Bath, a frequenter of pilgrimages) was not easy, as this fifteenth-century advice for travelers attests:

William Wey's Itinerary.

Compared to the trip to Jerusalem, the voyage to Compostella was brief, but it was by no means easy:

A Sea Voyage to Compostella

Such credit accrued to those who made such journeys that professional pilgrims were soon making the journey, returning with relics, badges, and pilgrim symbols (such as the palm for one who had made the trip to Jerusalem) and often with tall tales of the places they had visited. Chaucer's House of Rumor (in The House of Fame) characterizes pilgrims with "wallets stuffed with lies:

And, Lord, this hous in alle tymes
Was ful of shipmen and pilgrimes,
With scrippes bret-ful of lesinges.
(House of Fame 2121-23)

Langland says much the same in the Prologue to his Piers Plowman.

Pilgrims and palmers · pledged them together 
To seek Saint James · and saints in Rome.
They went forth on their way · with many wise tales, 
And had leave to lie · all their life after.
I saw some that said · they had sought saints;
Yet in each tale they told · their tongue turned to lies
More than to tell truth · it seemed by their speech.

The abuses Langland describes were fairly common; fake pilgrims were suitably punished:

False pilgrim condemned to the pillory.

For many (including, apparently, most of Chaucer's pilgrims) a pilgrimage was more a holiday, complete with sightseeing at the shrine; this is the case in The Prologue to The Tale of Beryn where the pilgrims spend much time in acting like tourists and no time in prayer.

It is not therefore surprising that moralists of the time, especially the Lollards, strongly objected to pilgrimages. The Lollard William Thorpe's description of pilgrimages sounds very much like Chaucer's, complete with bagpipes and bells on the horses: William Thorpe on Pilgrimages.