Agent Ransack should be able to search normal DOCX files just fine. Make sure you have the 'Office formats' option checked in the Options tab and the DOCX file you're searching is not protected.
To diagnose the problem you can use FileLocator Pro to view the text that is extracted from the document by looking at the 'Text' tab.
More information
To search DOC and DOCX files Agent Ransack uses something called IFilters. IFilters convert documents into text so that search programs can scan the contents. The latest IFilters can be downloaded from the Microsoft site:
http://www.microsoft.com/download/en/details.aspx?id=17062
However, Agent Ransack will only use the installed IFilters if the 'Office formats' check box is checked:

To check that the IFilters are working properly check the Configuration->Extensions->IFilters Settings for the specified file type.

If the file type, e.g. DOCX, does not appear and you've installed the Microsoft IFilter pack check that you're running the correct version of Agent Ransack for your OS architecture, e.g. 64-bit Agent Ransack on 64-bit windows.