Nintex is a powerful and flexible tool, but too much complex for business users (non-IT's). It uses a proprietary and outdated notation for process modeling, instead of the BPMN standard. This requires training for users.
Small processes are fine, but larger ones are really difficult to manage with this notation. This prevents rapid development of workflow applications and business users from creating new processes and evolving existing ones.
The workflow builder is very powerful, but also very complex. It is designed for IT staff or programmers. Lots of information and configuration in a myriad of submenus, attributes and options; to configure simple conditions in the flow you have to understand programming concepts such as variables, etc. Very error-prone and time-consuming, which prevents quick implementations and reduces ROI. Difficult to maintain in the long term.
Nintext's user interface is unfriendly, transaction-oriented and with a table-like design. It could generate difficulties to adopt the system, adding to the natural resistance to new tools, an old user interface.
Nintex's licensing model and its high price, avoids being able to perform proofs of concept and show results quickly. In addition, it involves a very relevant investment in the long term, considering the limitation in the number of processes that can be deployed (it requires additional payments to automate more processes).