He wants each rail franchise to be run by joint management teams, including representatives from both the train operating company and Network Rail, reports The BBC.
Mr Grayling said: “I intend to start bringing back together the operation of track and train on our railways.”
The changes will start when each franchise is renewed in the future.
The minister said he wanted the changes to improve services for passengers, who are travelling on an increasingly crowded and expensive network.
“We need to change the relationship between the tracks and the trains on the railway,” Mr Grayling said.
“In my experience passengers don’t understand the division between the two.
“They just want someone to be in charge. They want their train to work. I agree with them,” he added.
Each franchise will be run by one joint team, but the franchise owners and Network Rail will continue to exist separately.
The first new joint management teams will come into operation when the South Eastern and the East Midlands franchises are re-let in 2018.
The rail privatisation project was initiated in 1993 by John Major’s Conservative government.
The old state-owned British Rail was broken up, and the ownership of the UK’s rail infrastructure was separated from the control of the trains and services running on it.
This separation is held by some critics to be a significant source of delays to management decisions, repairs and train services.
Network Rail, which runs the UK rail infrastructure, was reclassified as a public sector body in September 2014.
Network Rail’s chief executive, Mark Carne, said he welcomed the new plan “to bring more joined up working within the industry”