Search results
1 de ene. de 2023 · Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits. Ruibo Liu, Chenyan Jia, Ge Zhang, Ziyu Zhuang, Tony X Liu, Soroush Vosoughi. We present Second Thought, a new learning paradigm that enables language models (LMs) to re-align with human values.
Second Thoughts Are Best: or, a Further Improvement of a Late Scheme to Prevent Street Robberies is a 1729 pamphlet by Daniel Defoe. He wrote it under the name of Andrew Moreton Esq., presented as a dissatisfied middle-class old man who was extremely concerned about the increase in criminality around the 1720s.
arXiv:2301.00355v2 [cs.CL] 5 Jan 2023 Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits Ruibo Liu1, Chenyan Jia2, Ge Zhang3,4, Ziyu Zhuang1∗, Tony X. Liu 2, Soroush Vosoughi1 1DartmouthCollege, 2Stanford University, 3Beijing Academy of Artificial Intelligence,4University of Michigan,Ann Arbor 1{ruibo.liu.gr, soroush.vosoughi}@dartmouth.edu
Trained with SECOND THOUGHTS, LMs can not only re-align their generation with human values, even when the context has already been poisoned, but also show the chain of editing steps for ease of interpretability and to facilitate further edits (§4.5).
Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits. Ruibo Liu1, Chenyan Jia2, Ge Zhang3, Ziyu Zhuang1∗, Tony X. Liu 2, Soroush Vosoughi1 1Dartmouth College,2Stanford University,3University of Michigan, Ann Arbor. 1{ruibo.liu.gr, soroush.vosoughi}@dartmouth.edu. Abstract.
to change your opinion about something or start to doubt it: You're not having second thoughts about getting married, are you? on second thoughts UK (US on second thought) used when you want to change a decision you have made: Can I have a cup of coffee, please? - actually, on second thoughts, I'll have a beer.
Abstract. We present Second Thoughts, a new learning paradigm that enables language models (LMs) to re-align with human values. By modeling the chain-of-edits between value-unaligned and value-aligned text, with LM fine-tuning and additional refinement through reinforcement learning, Second Thoughts not only achieves superior performance in ...
