Time.
When books are converted into film the biggest challenge is time. The average length of a movie is about 2 hours; most books that are adapted into movies are at least 300 pages. It is very hard to condense a 300+ page book into two hours and a lot of details that are significant to readers get cut. Take Harry Potter for example, several characters’ well developed back story gets eliminated in the films. Certain character get eliminated altogether (Peeves, Charlie Weasley, Teddy Lupin, Professor Binns, Collin Creevy after CoS and Dennis Creevy, Dobby between CoS and DH, Kreacher, Winky, The Gaunts, etc.). We also lost out on many scenes like the ones that occur in OotP at St. Mungo’s Hospital for Magical Maladies and Injuries, where we learn about what happened to Neville’s parents; or parts of scenes like in PS/SS with the centaurs, other than Firenze (Bane, Magorian, etc.), where we first see that the centaurs believe that they are better than humans. As well the characters do not end up being nearly as complex in the films as they are in the book. In the Harry Potter films Ron and Neville seem to only serve the purpose of being comic relief whereas in the books they are much more complex. We also lose a lot of Harry’s personality between the pages and the screen. In the books, Harry has a tendency to be sarcastic and that is lost in the first few films and we just barely see some of that aspect of his personality emerge in the films for OotP, HBP and DH. The reason that most film adaptations do not live up to their literary counterparts is because of the loss of complexity that the lack of time forces on the telling of the story.
PS/SS- Philosopher’s Stone/ Sorcerer’s Stone(book 1)
CoS- Chamber of Secrets(book 2)
PoA- Prisoner of Azkaban(book 3)
GoF- Goblet of Fire(book 4)
OotP- Order of the Pheonix(book 5)
HBP- Half-Blood Prince(book 6)
DH- Deathly Hallows(book 7)