Plans for a gravel extraction quarry in Wallingford which have been refused twice by the council have now been appealed to the Secretary of State.