The aim of answering 80% of calls within five minutes would still see 16 million callers waiting longer than that to be answered by HM Revenue and Customs, its report said.
The Public Accounts Committee said HMRC faced more calls but had fewer staff.
HMRC is to close its enquiry centres to focus on phone and online assistance, reports The BBC.
It announced the move last week.
The committee said that the tax authority received 79 million calls in 2011-12 about issues such as self-assessment forms, but 20 million of these calls were not answered.
A third of the letters received by HMRC in the same year were not given a response within a 15-day target.
Margaret Hodge, who chairs the committee, described this record as “abysmal” but welcomed HMRC’s changing attitude to customer service.
“Officials are beginning to realise that good customer service lies at the heart of any strategy to maximise revenues while cutting costs,” she said.
She welcomed the fact that call costs were to be reduced by ending the use of 0845 numbers later in the year.
But the committee said that a target of answering 80% of calls within five minutes was still well short of the industry benchmark of 80% of calls being answered within 20 seconds.
“While achieving this target would be an improvement on current performance, it remains unambitious and woefully inadequate,” the committee said.
It added that the introduction of new benefits meant more calls were likely to be made to HMRC in the near future, but HMRC planned to cut a third of its customer-facing staff.
On Thursday, HMRC announced it was to close all of its 281 enquiry centres in 2014. These offices gave face-to-face help to 2.5 million people with tax queries last year.
People will need to contact the phone line or go online to get their tax query answered. Advisers may then decide that the issue should be discussed face-to-face at the caller’s home or elsewhere, under the plans, which HMRC said would save £13m a year.